ReverB-SNN: Reversing Bit of the Weight and Activation for Spiking Neural Networks
Authors: Yufei Guo
Yuhan Zhang
Zhou Jie
Xiaode Liu
Xin Tong
Yuanpei Chen
Weihang Peng
Zhe Ma
Venue: ICML 2025 poster (2025)
Keywords: spiking neural network
Primary area: applications->neuroscience_cognitive_science
TL;DR: Reversing Bit of the Weight and Activation for Spiking Neural Networks

Abstract: The Spiking Neural Network (SNN), a biologically inspired neural network infrastructure, has garnered significant attention recently. SNNs utilize binary spike activations for efficient information transmission, replacing multiplications with additions, thereby enhancing energy efficiency. However, binary spike activation maps often fail to capture sufficient data information, resulting in reduced accuracy.
To address this challenge, we advocate reversing the bit of the weight and activation, called \textbf{ReverB}, inspired by recent findings that highlight greater accuracy degradation from quantizing activations compared to weights. Specifically, our method employs real-valued spike activations alongside binary weights in SNNs. This preserves the event-driven and multiplication-free advantages of standard SNNs while enhancing the information capacity of activations.
Additionally, we introduce a trainable factor within binary weights to adaptively learn suitable weight amplitudes during training, thereby increasing network capacity. To maintain efficiency akin to vanilla \textbf{ReverB}, our trainable binary weight SNNs are converted back to standard form using a re-parameterization technique during inference.
Extensive experiments across various network architectures and datasets, both static and dynamic, demonstrate that our approach consistently outperforms state-of-the-art methods.

ReverB-SNN: Reversing Bit of the Weight and Activation for Spiking Neural Networks
Yufei Guo Yuhan Zhang Zhou Jie Xiaode Liu
Xin Tong Yuanpei Chen
Weihang Peng Zhe Ma
Abstract
et al., 2016; Ming et al., 2023), object segmentation (Ron- neberger al., 2015), and object tracking (Bewley et al., 2016). However, to achieve enhanced performance, network architectures are becoming increasingly complex (Huang et al., 2017; Devlin et al., 2018). Several techniques have been proposed to tackle this complexity, including quantiza- tion (Gong et al., 2019), pruning (He et al., 201 knowl- edge distillation (Hinton et al., 2015; Polino et al., 2018; Zhang et al., 2022), and the emergence of spiking neural networks (SNNs) (Maass, 1997; Li et al., 2021a; Xiao et al., 2021; Wang et al., 2022; Bohnstingl et al., 2022; Yu et al., 2025; Guo et al., 2025; Yao et al., 2023; Guo et al., 2023a). SNNs, touted as the next-generate gext-rennrttion neural networks, re- duce energy consumption by emulating brain-like informa- tion processing through spike-based communication, which translates weight and activation multiplications into addi- tions, facilitating multiplication -fee inference. Moreover, their event-driven computational model demonstrates su- perior energy efficiency neuromorphic hardware plat- forms (Ma et al., 2017: Akopyan et al., 2015; Davies et al., 2018; Pei et al., 2019).
The Spiking Neural Network (SNN), a biologi- cally inspired neural network infrastructure, has garnered significant attention recently. SNNs uti- lize binary spike activations for efficient informa- tion transmission, replacing multiplications with additions, thereby enhancing energy efficiency. However, binary spike activation maps often fail to capture sufficient data information, resulting in reduced accuracy. To address this challenge, we advocate reversing the bit of the weight and acti- vation for SNNs, called ReverB-SNN, inspired by recent findings that highlight greater accuracy degradation from quantizing activations compared to weights. Specifically, our method employs real- valued spike activations alongside binary weights in SNNs. This preserves the event-driven and multiplication-free advantages of standard SNNs while enhancing the information capacity of acti- vations. Additionally, we introduce a trainable fac- tor within binary weights to adaptively learn suit- able weight amplitudes during training, thereby increasing network capacity. To maintain effi- ciency akin to vanilla ReverB-SNN, our trainable binary weight SNNs are converted back to stan- dard form using a re-parameterization technique during inference. Extensive experiments across various network architectures and datasets, both static and dynamic, demonstrate that our approach consistently outperforms state-of-the-art methods.
However, it has been observed that SNNs' binary spike activation maps suffer from limited information capacity, failing adequately capture membrane potential informa- tion during quantization, thereby diminishing accuracy (Guo et al., 2022d;a; Wang et al., 2023; Sun et al., 2022). To ad- dress this, some studies have explored alternatives such as ternary spikes (Sun et al., 2022), integer spikes (Wang et al., 2023; Fang et al., 2021b; Feng et al., 2022), and even real-valued spikes (Guo al., 2024c;d; 2025), how- ever these approaches often come at the cost of increased energy consumption due to the inability to convert weight and activation multiplications into additions.
1. Introduction
Recent research (Gong et al., 2019; Qin et al., 2024) in- dicates that using low-bit weights in ANNs can achieve higher accuracy compared to low-bit activations. Motivated by these findings, this paper proposes a novel approach to enhance spike activation's information capacity while preserving the advantages of multiplication-free and event- driven SNNs. Specifically, unlike the conventional binary spike activation approach, we advocate for real-valued spike activations similar to EGRU (Subramoney et al., 2023) to
Artificial Neural Networks (ANNs) are currently extensively applied across various fields such as object recognition (He
Intelligent Science& & Technology Academy of CASIC, China. Yufei Guo and Yuhan Zhang contributed equally to this study. Correspondence to: Yufei Guo syfguo@pku..duu.oo Zhe Ma <mazhe_thue 163.com>.
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Reversing Bit of the Weight and Activation for Spiking Neural Networks
Vanilla SNN
Real-valued Weights
Binary Activation
Results
00000100
Our SNN
Binary Weights
Real-valued Activation
Results
00000100
Figure 1. The difference between our SNN and the vanilla SNN. Our SNN differs significantly from the vanilla SNN. The vanilla SNN employs binary spikes, leading significant information loss of the activations. In contrast, SNN utilizes real-valued spikes alongside binary weights, therehy enhancing the neuron's 's information capacity. This approach retains the benefits of event-driven processing and multiplication-adddtton transformations.
increase information capacity. Correspondingly, we adapt realevalued weights to binary weights {-1,1}, ensuring retention of multiplication-free and event-driven benefits. Recognizing that binary weights may limit network capac- ity, we extend them to learnable form {-a,at a),, where a is a learnable parameter. During inference, we introduce a re- c--arameterization technique to integrate the a factor into the activation process, thereby preserving the multiplication- free inference capability still.
We evaluate our methods on both static datasets (CIFAR-10 (Krizhevsky al., 2010), CIFAR- 100 (Krizhevsky et al., 2010), ImageNet (Deng et al., 2009)) and spiking datasets CCIAARI-DDY (Li et al., 2017)) using widely adopted backbones. Results demonstrate the effectiveness and efficiency of our approach. For instance, using ResNet34 with only timesteps, our method achieves a top-1 accuracy of 70.91% on ImageNet, representing 3.22% improve- other state-of-the-art SNN models.
The distinction between SNN and the conventional SNN is illustrated in Fig. 1. In summary, our contributions can be summarized as follows:
2. Related Work
In this section, we provide a brief overview of recent ad- vancements in SNNs focusing on two key aspects: learning methods and information loss reduction techniques.
We advocate enhancing the information capacity of spike activations by employing real-valued spikes alongside binary weights in SNNs. This approach preserves the multiplication-fret and event-driven ad- vantages of standard SNNs while introducing novel paradigm with realevalued spike neurons and binary weights.
2.1. Learning Methods of Spiking Neural Networks
There are primarily two approaches to achieving high- performance deep SNNs. The first approach involves con- verting well-trained ANN into an SNN, known as ANN- SNN conversion (Han Roy, 2020: Kim al., 2020: Han et al., 2020; Liu al., 2022: Yu et al., 2021). This method maps parameters from pretrained ANN to its SNN counter- part by aligning ANN activation values with SNN average firing rates. Despite its widespread use due to the resource efficiency compared to training SNNs from scratch, ANN- SNN conversion has inherent limitations. constrained by rate-coding schemes and overlooks the temporal dynam-
Additionally, we propose variant with learnable bi- nary weights and re-paar oe-paaameterization technique. Dur- ing training, the weight magnitude is learned, and during inference, this magnitude is folded into the activation via re-parameterization. This ensures that the binary weights {-a, a) revert to standard binary weights {-1,1}, maintaining the addition-onle only advan- tage.
2

---

Reversing Bit of the Weight and Activation for Spiking Neural Networks
ies unique SNNs, limiting its efficacy with neuromor- phic datasets. Moreover, achieving comparable accuracy to ANNs typically requires numerous timesteps, increasing energy consumption contrary to SNN's low-power design intent. Additionally, SNN accuracy often falls short of ANN accuracy, constraining SNN's potential and research value.
overcome the challenge. Finally, we propose variant with learnable binary weights aimed at further enhancing network capacity.
3.1. Preliminary
The spike neuron, inspired by the brain's functionality, serves the fundamental and distinctive computing unit within SNNs. It closely mimics the behavior of biological neurons, characterized by the interplay between membrane potential and input current dynamics. In this paper, we focus on the widely used model, which is governed by the iterative equation (Wu et al., 2019):
Training SNNs directly from scratch. particularly suited for neuromorphic datasets, has gained attention for its efficiency in reducing timesteps, sometimes to fewer than 5 (Guo et al., 2022d; Fang et al., 202 la; Wu et al., 2018; Rathi & Roy, 2020; Wu et al., 2019; Neftci et al., 2019; Ren et al., 2023). Another emerging approach is hybrid learning, which blends the benefits of ANN-SNI conversion and direct training methods (Rathi & Roy, 2020; Wu et al., 2021a; Zhang et al., 2024; Guo et al., 2023b; 2024d). This approach has also gareered significant interest. In this paper, focus enhancing the performance of directly trained SNNs by ad- dressing information loss, an area underexplored in existing literature.
(LIF) neuron
-
Uf< < Vth-
(I)
Here, Uj represents the membrane potential time-step ( for the 1-th layer, 0 denotes the spike output from the preceding layer, W denotes the weight matrix at the (-th layer, Vta is the firing threshold. and T is the time constant governing the leak in the membrane potential. When the membrane potential surpasses the firing threshold, the neu- ron emits a spike and resets to its resting state, characterized by:
2.2. Information Loss Reducing Methods of Spiking Neural Networks
Several methods aim to mitigate information loss of the ac- tivation in SNNs by altering spike activation precision (Guo et al., 2022c;a;;; Wang et al., 2023). For instance, a ternary spike neuron transmitting information via (,, 1, spikes was proposed in (Sun et al., 2022), which enhances infor- mation capacity but lacks the iiaippiicaatonaadittio trans- formation advantage. Then, a new method was improved upon this with a ternary spike using {-1, 0, }) values, main- taining both improved activation information capacity and the multiplication-aduu addiiioo advantage (Guo et al., 2024b). In MT-SNN (Wang et al., 2023) and DS-ResNet (Feng al., 2022), a multiple threshold (MT) algorithm was in- troduced for Leak- Seak--ntegrate-Fiin
of o
J 0, otherwise
1, if Uf 2 Vin
While the binary spike-based processing paradigm is energy- efficient, it suffers from suboptimal task performance due to limitations in information representation. This motivates our exploration of alternative approaches to enhance the information capacity of SNNs.
Classifier in SNNs. In a classification model, the final output is typically processed using the Softmax function to predict the desired class object. In the context of SNN models, a common approach, as seen in recent studies (Guo et al., 2022c;d; Fang et al., 2021c), involves aggregating the outputs from all time steps to obtain the final output:
(LIF) neurons, allowing omission of integer spikes to enhance information transfer. SEWNet (Fang et al., 2021b) proposed an integer spike format by modifying the position of the shortcut module. Some approaches employ real-valued spikes directly to sig- nificantly boost information capacity (Guo et al., 2024c; 2025). Nevertheless, these above works all are albeit at the cost of increased energy consumption.
T
Ovut Σ
O
out
(3)
Subsequently, the cross-entropy loss is computed using the true label and Softmax(Oauu).
This paper explores the adoption of real-valued spike activa- tion while preserving a multiplication-fie free advantage using binary weights.
3.2. ReverB: Reversing the Bit of Weight and Activation
To address the issue of information loss in activation, we introduce the ReverB-SNN method, inspired by recent re- search highlighting greater accuracy degradation from quan- tizing activations compared to weights (Gong et al., 2019; Qin et al., 2024). Specifically, we employ real-valued spike activations, where the output spike at time for the I-th layer is defined follows:
3. Methodology
In this section, we begin by introducing the fundamentals of SNN to illustrate its method of information processing and its inherent limitations in information loss. Subsequently, we introduce our ReVerB-SNN method as a solution to
0 - U% if ir 2 Vun
(4)
0. otherwise
3

---

Reversing Bit of the Weight and Activation for Spiking Neural Networks
Meanwhile, to maintain the multiplication-free and event- driven advantages of standard SNNs, the real-valued weights are converted to binary weights. Consequently, the membrane potential dynamics are updated as:
methods used for managing firing activity, we adopt Straight Through Estimator (STE) (Rathi & Roy, 2020; Guo et al., 2022c) gradients to tackle this problem. Mathematically, STE surrogate gradients are defined as:
= + O
Uf < Vin
(5)
dWo 1, - 1
- 0, otherwise
(10)
dW
where Vh sign ()) = +1, if W220
1, otherwise Thii ap-
This approach allows us to manage the non-differentiabiliy inherent in weight binarization within our SNN framework effectively.
proach ensures that activations remain real-valued while leveraging binary weights for computational efficiency, thereby mitigating accuracy loss in SNNs.
Training of Our Method. In our study, we employ the backpropagation (STBP) algorithm (Wu et al., 2019) to effectively train our SNN models. STBP treats the SNN as a self-recurrent neural network, facilitating error backpropagation akin to principles used in Convolu- tional Neural Networks (CNNs). The gradient at layer I, derived through the chain rule, is expressed as:
Event-driven Advantage Retaining, The event-driven sig- nal processing characteristic of SNNs significantly enhances energy efficiency. Specifically, a spiking neuron will only emit signal and initiate subsequent computations if its membrane potential exceeds the firing threshold, Vuai oth- erwise, it remains inactive. Similarly, our real-valued spike neuron also benefits from this event-driven property. It acti- vates and emits a real-valued spike to initiate computations only when its membrane potential exceeds Vtn.
aL OW. Σ(
aL a0; ØL aUi+1 au aw; aof oue
oUtt1 oUB
ow ow,"
(11)
Addition-only Advantage Retaining. The ability of SNNs to use addition instead of multiplication contributes signifi- cantly to their energy efficiency. In a standard binary spike neuron, when a spike is fired, it traditionally multiplies a weight, w, connected to another neuron to transmit informa- tion, expressed as:
aw, where aw, is surrogate gradient for the binarization of the weight in 1-th layer. This approach enables us to train SNNs effectively by propagating errors through time and across network layers, leveraging the benefits of both temporal and spatial information in neural processing.
T Wt
(6)
3.3. Learnable Binary Weight Variant
where w E RR. Given the spike amplitude is 1, this multipli- cation simplifies to an addition:
As mentioned earlier, while real-valued activations increase information capacity, binary weights can decrease network capacity. To address this issue, we extend binary weights to a learnable form, not restricted to {-1, but rather {-a, at where a is a learnable parameter defined as:
x = 0 + W.
(7)
In our real-valued spike neuron, although the spike is real- valued, the weight w² B is binary, and the multiplication of and w can be represented as:
+1 a, if W W trainable = a sign(W) = -1 -1-a, a, otherwise
W
t = ox x 1, or, 0 -1
(8)
(12)
Introducing allows weights to adapt their amplitude. This parameter a is applied in a channel-wise manner across our SNN models. Consequently, the membrane potential dynamics are adjusted to:
This too can be simplified to an addition operation:
x 0 0, or, 0 0-o.
In summary, our proposed method enhances the activation expression capability of SNNs while preserving the event- driven and addition-only advantages of traditional SNNs.
U = - + w Wi.timimblle O'
Uf < Vin (13)
Regarding gradients, the gradient of W, at the layer I is given by:
Surrogate Gradient for Weight Binarization. In vanilla SNNs, the firing behavior of spiking neurons is non- differentiable, necessitating the use of surrogate gradient (SG) methods in many studies (Rathi & Roy, 2020; Guo et al., 2022c) to address this issue. In our SNN framework, while the firing activity of spiking neurons becomes differ- entiable, the process of weight binarization poses a non- differentiability challenge. Therefore, similar to other SG
aL aw,
Σ OL d0t + ao aue au/41 aue ow aw,
aL out aut ow?
(14)
While the gradient of an at the layer is:
ØL Jou
OL a0t
ØL au aut
Σ
(15)
aof तhe
ohA oue don

---

Reversing Bit of the Weight and Activation for Spiking Neural Networks
Algorithm I Training and inference of SNN.
Dataset
Method
Time-step
Accuracy
Training
Vanilla SNN
2
92.80%
Input: An SNN be trained where the precision of weights and activations reversed; training dataset; total training
ReverB
2
94.14%
Learnable variant
94.45 % 2
iteration: Irain-
CIFAR-10
Vanilla SNN
4
93.85%
Output: The trained SNN.
ReverB
4
94.55 %
for all = 1, Irain iteration do
Learnable variant
4
94.96 %
Get mini-batch training data, (t) and class label, y()::
Vanilla SNN
2
70.18%
ReverB
2
72.54 %
3: Feed the win (2) into the SNN and calculate the SNN output, (i) by Eq.3,
Learnable variant
2
72.95 %
CIFAR-100
Vanilla SNN
4
71.77%
4: Compute classification
loss
LCE
ReverB
4
72.93% %
(Oul (f), y());
Learnable variant
4
73.28% %
5: Calculate the gradient w.r.t. W by Eq. 14 and the gradient w.r.t. a by Eq. 15;
6: Update W: (W W (aw) and (a
Table 1. Ablation study for the ternary spike on CIFAR.
195)) where is learning rate.
1)
7: end for
Re-parameterization Technique. To maintain computa- tional efficiency in SNNs during inference, we propose a re- parameterizzatio technique. Obviously, the Eq. 16 can be further written as
Re- p-raammtterization
Input: The trained SNN with trainable weights and real-valued spikes ; total layer of SNN:
Output: The re-parameterized trained SNN with- normalized binary weight and real-valued spikes.
= - +f
Uf Van
(16)
To convert the real-valued weight W back to binary ef- fectively during inference, we fold the a into the out- put 0 of the previous layer as a new output, defining O new, 1-1 = a00 This adjustment simplifies Eq. 16 to:
1: for all i = 1, number do
2:
Fold the parameters a; into firing function by Eq. 18;
3: end for
Inference
Uf - W new,1-1 Uf Vin.
(17)
Input: The re-parameterized trained SNN; test dataset; total
test iteration: Test-
Thus the real-valued weight will be converted to the binary weight again. In this way, the output spike at time t for I layer is updated as follows:
Output: The output.
1: for all =
4 iteration do
2: Get mini-batch test data, æin (i) and class label, yii)
O Ouew, = aUi- if ff Vtn
in test dataset;
(18)
0, otherwise
3: Feed the am (i) into the reparameterized SNN and
calculate the SNN output, (i) )) by Eq.
Thereby, the multiplication the weight and the activation could be converted addition again the inference.
4: Compare the classification factor Oout (t) and y(i)
for classification.
5: end for
In summary, by embedding learnable factor a into the weight during training, enhance the network capacity. During inference, we extract this factor from the weight and fold it into the output spike of the previous layer. This approach allows us to maintain the advantages of normalized binary weights and real-valued spikes the trained SNN, without altering the neuron's 's update process.
Since the W traiuble and O are both real-valued in our SNN, using trainable weights introduces the challenge that the multiplication of weight and activation cannot be trans- formed into an addition, potentially losing the computa- tional efficiency advantages of SNNs. To address this, we decoupling technique. This
For detailed outline of the training and inference processes of our SNN, refer to Algorithm 1.
propose training-inferent method converts different amplitude weights into a nor- malized binary form during the inference phase through re- prraaatttrriztionn, ensuring retention the multiplication- free efficiency advantages.
4. Experiments
We conducted comprehensive experiments to assess the effectiveness of the proposed ReverB-SNN method and
5

---

Reversing Bit of the Weight and Activation for Spiking Neural Networks
Dataset
Method
Type
Architecture
Timestep
Accuracy
SpikeNorm (Sengupta al., 2019)
ANN2SNN
VGG16
2500
91.55%
Hybrid-Train (Rathi al., 2020)
Hybrid training
VGGI6
200
92.02%
TSSL-BP (Zhang & Li, 2020)
SNN training
CIFARNet
5
91.41% %
TL (Wu al., 2021b)
Tandem Learning
CIFARNet
8
89.04%
PTL (Wu et al., 2021c)
Tandem Learning
VGG11
16
91.24%
PLIF (Fang et al., 2021c)
SNN training
PLIFNet
8
93.50%
DSR (Meng et al., 2022)
SNN training
8
20
95.40%
KDSNN (Xu et al., 2023)
SNN training
ResNet18
4
93.41% %
91.78%
Diet-SNN (Rathi Roy, 2020)
SNN training
ResNet20
92.54% 10
CIFAR-I0
93.13%
Dspike (Li et al., 2021b)
SNN training
ResNet20
93.66% 4
92.34%
STBP-ddNN (Zheng et al., 2021)
SNN training
ResNet19
92.92%
94.16%
TET (Deng et al., 2022)
SNN training
ResNet19
94.44% 4
93.64%
RecDis-SNN (Guo et al., 2022c)
SNN training
ResNet19
95.53% 4
95.31%
ResNet19
Real Spike (Guo et al., 2022d)
SNN training
95.51% 4
ResNet20
91.89% 4
95.97% 10.08
ResNet19
2
96.39 10.11
ReverB
SNN training
2
94.14% 10.08
ResNet20
4
94.55%
1
96.22 % 10.12
ResNet19
2
96.62 % 10.11
Learnable variant
SNN training
2
94.45 % +0.07
ResNet20
4
94.96% % 10.10
RMP (Han et al., 2020)
ANN2SNN
ResNet20
2048
67.82%
Hybrid-Train (Rathi et al., 2020)
Hybrid training
VGG11
125
67.90%
T2FSNN (Park et al., 2020)
ANN2SNN
VGG16
680
68.80%
Real Spike (Guo et al., 2022d)
SNN training
ResNet20
5
66.60%
LTL (Yang et al., 2022)
Tandem Learning
ResNe120
31
76.08%
Diet-SNN (Rathi & Roy, 2020)
SNN training
ResNet20
64.07% 5
CIFAR- 100
RecDis- (Guo et al., 2022c)
SNN training
ResNet19
74.10% 4
2
71.68%
Dspike (Li et al., 2021b)
SNN training
ResNet20
73.35% 4
72.87% 2
TET (Deng et al., 2022)
SNN training
ResNet19
74.47% 4
77.62% 10.10
ResNet19
2
1..3%%10.1.
ReverB
SNN training
2
72.54 % 10.09
ResNet20
72.93% 10.12
78.06 % 10.08
ResNet19
2
78.46 % +0.12
Learnable variant
SNN training
2
72.95% +0.11
ResNet20
73.28 % +0.08
Table 2. Comparison with SoTA methods CIFAR-10(100).
6

---

Reversing Bit of the Weight and Activation for Spiking Neural Networks
Method
Type
Architecture
Timestep
Accuracy
STBP-dBBN (Zheng et al., 2021)
SNN training
ResNet34
63.72%
TET (Deng et al., 2022)
SNN training
ResNet34
64.79%
RecDis-SNN (Guo et al., 2022c)
SNN training
ResNet34
67.33%
OTTT (Xiao et al., 2022)
SNN training
ResNet34
65.15%
GLIF (Yao et al., 2022)
SNN training
ResNet34
67.52%
DSR (Meng al., 2022)
SNN training
ResNet18
50
67.74%
Temary spike (Guo et al., 2024b)
SNN training
ResNet34
70.12%
SSCL (Zhang et al., 2024)
SNN training
ResNet34
66.78%
TAB (Jiang et al.)
SNN training
ResNet34
67.78%
MPBN (Guo et al., 2023c)
SNN training
ResNet34
64.71 %
Shortcut back (Guo et al., 2024a)
SNN training
ResNet34
67.90%
Multi-hierarchical model (Hao al., 2023)
SNN training
ResNet34
69.73%
SML (Deng al., 2023)
SNN training
ResNet34
68.25%
ResNet18
63.68%
Real Spike (Guo al., 2022d)
SNN training
ResNet34
67.69%
ResNet18
63. 68%18%
SEW ResNet (Fang et al., 202
SNN training
ResNet34
67.04%
ResNet18
66.22 % 100.16
ReverB
SNN training
ResNet34
70.74 1.0.1
ResNet18
66.58% 10.14
Learnable variant
SNN training
ResNet34
70.91 #0.13
Table 3. Comparison with SoTA methods on ImageNet.
its learnable binary weight variant. Our evaluation in- cluded comparisons with several SoTA methods across a range of widely recognized architectures. Specifically, we tested spiking ResNet20 (Rathi & Roy, 2020; Sengupta al., 2019) and ResNet19 (Zheng et al., 2021) on CIFAR- 10(100) (Krizhevsky et al., 2010), ResNet18 (Fang al., 2021a) and ResNet34 (Fang et al., 2021a) ImageNet, as well as ResNet20 and 19sNet19 CIFARIO-DVS (Li et al., 2017).
with different timesteps. The results are summarized in Table 1.
The baseline accuracy of vanilla ResNet20 with 2 timesteps reaches 92.80% and 70.18% on CIFAR-10 and CIFAR-100 respectively, consistent with previous studies. Implement- ing the ReverB-SNN method significantly improves perfor- mance to 94.14%% and 72.54% respectively, marking sub- stantial enhancements of approximately 1 1.30% and 2.50%. Furthermore, integrating the learnable binary weight vari- ant leads to additional performance gains, resulting in final accuracies of 94.45 44.45% for CIFAR-10 10 and 72.95% for CIFAR- 100. These findings underscore the efficacy of our approach in enhancing model performance. When the model is evalu- ated with 4 timesteps, method continues to demonstrate its effectiveness. The performance improvements observed with this configuration further validate the robustness and efficacy of the ReverB-SNN technique, underscoring its potential for enhancing model accuracy across various set- tings.
In our work, we used the SGD optimizer to train our mod- els with a momentum of 0.9 and learning rate of 0.1, which decays to 0 following cosine schedule. For the CIFAR 10000) and CIFAR-DVS datasets, trained the models for 400 epochs with a batch size of 128. On Ima- geNet, we trained for 300 epochs with the same batch size. Data augmentation was performed using only flip opera- tion. The train and test splits follow the settings provided by the official dataset. The membrane potential decay constant is set 0.25, In these static datasets, Vih 0 all the time since static datasets can not provide timing information. For neuromorphic datasets, set it to 0.25.
4.2. Comparison with SoTA Methods
In this section, we conducted a comparative analysis of our approach against SoTA methods. We present the top-1 ac- curacy results along with the mean accuracy and standard deviation derived from trials. Our evaluation focused initially on the CIFAR- 10 and CIFAR-100 datasets. The summarized results are presented in Table 2. For the CIFAR-
4.1. Ablation Study
We conducted series of ablation experiments evaluate the effectiveness of the proposed ReverB-SNN method and its learnable binary weight variant the CIFAR-10 and CIFAR-100 100 datasets, employing ResNet20 as the backbone

---

Reversing Bit of the Weight and Activation for Spiking Neural Networks
Method
Type
Architecture
Timestep
Accuracy
DSR (Meng et al., 2022)
SNN training
VGG11
20
77.27%
GLIF (Yao et al., 2022)
SNN training
7B-wideNet
16
78.10%
STBP-ddB (Zheng et al., 2021)
SNN training
ResNet19
10
67.80%
RecDis-SNN (Guo et al., 2022c)
SNN training
ResNet19
10
72.42%
Real Spike (Guo et al., 2022d)
SNN training
ResNer19
72.85% 10
Dspike (Li al., 2021b)
SNN training
ResNer20
75.40% 10
Spikformer (Zhou et al., 2023)
SNN training
Spikformer
78.90% 10
SSCL (Zhang et al., 2024)
SNN training
ResNet19
80.00% 10
ResNet19
10
50.30%:0.0
ReverB
SNN training
ResNer20
10
77.80 % +00.0
ResNet19
10
(0.50%:1..0
Learnable variant
SNN training
ResNet20
10
78.10%
Table 4. Comparison with SoTA methods on CIFAR10-DVS.
10 dataset, previous methods achieved peak accuracies of 95.53% using ResNet19 and 93.66% using ResNet20 their backbone architectures. In contrast, our ReverB-SNN method achieves 96.39% and 94.55% respectively, while utilizing fewer timesteps. Furthermore, leveraging learn- able binary weights enables our SNN models to attain even higher accuracies. Moving to the CIFAR-100 dataset, our learnable binary weight variant applied to ResNer19 and ResNet20 could achieve superior performance with just 2 timesteps. Our method surpasses leading approaches like TET and RecDis-SNN by approximately 4.0% with ResNer19, despite these methods using 4 timesteps. These experimental findings underscore the efficiency and efficacy of our proposed methodology.
and ResNet20 backbone architectures, method achieved accuracies of 80.50% and 78. 10% respectively, transcending the 80% milestone for ResNet19 even. This marks a substantial improvement in performance on this widely used neuromorphic dataset.
5. Energy Estimation
In this section, evaluate the hardware energy cost asso- ciated with the vanilla SNN model and the ReverB-SNN model using ResNet20 on the CIFAR10 with 2 timesteps for single image inference. Since the first rate-encoding layer does not enjoy the multiplication-fvee property. it will pro- duce the FLOOPs (floating point operations). While other lay- ers are calculated by SOPs (synaptic operations). The SOPs are calculated by x T A, where is the mean sparsity, T is the timestep and A denotes the number of additions in the equivalent artificial neural network (ANN) model. For the Vanilla model, the sparsity of the SNN is 16.42%, whereas for the ReverB-SNN model, is 17.18%. We calculate energy consumption based on the methodology outlined in (Hu et al., 2021), where one FLOP consumes 12.5 pJ and one SOP consumes 77 fJ. A summary of the energy costs is provided in Table 5. The ReverB-SNN method results in only modest 0.52% increase in energy cost compared to the baseline vanilla model. This minimal increase highlights the efficiency of the ReverB-SNN approach, demonstrating that can achieve improved performance with a relatively small additional energy expenditure.
Method
Accuracy
#Flops
#Sops
Energy
Vanilla SNN ReverB
92.80%
3.54M
71.20M
49.73uJ
94.14%
3.54M
74.50M
49.99uJ
Table 5. Energy estimation.
In our subsequent experiments, we evaluated our approach on the challenging ImageNet dataset, renowned for its com- plexity compared CIFAR. Table presents the compar- ative results. Recent SoTA benchmarks on this dataset in- clude RecDis-SNN (Guo et al., 2022c), GLIF (Yao et al., 2022), DSR (Meng et al., 2022), Real Spike (Guo et al., 2022d), and SEW ResNet (Fang et al., 2021a), achieving ac- curacies of 67.33%, 67.5 07.52%, 67.74%, 67.69%, and 67.04% respectively. Our method achieves significantly higher ac- curacy, reaching up 70.91% 1,, 3.22% improvement over other SoTA SNN models. This notable improvement un- derscores the effectiveness of our approach for large-scale datasets.
6. Conclusion
This study has introduced ReverB-SNN, novel approach for enhancing SNNs by integrating real-valued spike ac- tivations with binary weights. Our method addresses the challenge of reduced aceuracy in SNNs due to limited in- formation capture by binary spike activation maps. By
In our final evaluation, we applied our SNN model to the CIFAR 10-DVS neuromorphic dataset. Utilizing ResNet19
8

---

Reversing Bit of the Weight and Activation for Spiking Neural Networks
reversing the bit of both weights and activations, have preserved the energy-efficiee and multiplication-free char- acteristics of traditional SNNs while significantly boosting the information capacity of activations. Moreover, the in- troduction of a trainable factor within binary weights has enabled adaptive learning of weight amplitudes during train- ing, thereby enhancing the overall network capacity. Im- portandly, to ensure operational efficiency comparable to standard SNNs, proposed re-parameterization tech- nique that converts trainable binary weight SNNs back to standard form during inference. Extensive experimental validation across diverse network architectures and datasets, encompassing both static and dynamic scenarios, consis- tently demonstrates the superiority of our approach over existing state-of-the-art methods.
L. Imagenet: A large-scale hierarchical image database.
In 2009 IEFE conference on computer vision and pattern recognition, pp. 248-255 Icee, 2009.
Deng, S., Li, Y., Zhang, S., and Gu, S. Temporal effi- cient training of spiking neural network via gradient re- weighting. arXiv preprint arXiv:2202. 1946, 2022.
Deng, S., Lin, H., Li, Y., and Gu, S. Surrogate module learn- ing: Reduce the gradient error accumulation in training spiking neural networks. In International Conference on Machine Learning, pp. 7645-7657. PMLR, 2023.
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. Bert: Pre-training of deep bidirectional transformers for lan- guage understanding. arXiv preprint arXiv: 1810.04805, 2018.
Acknowledgements
This work was supported by the National Key Research and Development Program of China (No. 2024YDLN0013) and the National Natural Science Foundation of China (No. 12202412).
Fang, W., Yu, Z., Chen, Y., Huang, T., Masquelier, T., and Tian, Y. Deep residual learning in spiking neural net- works. Advances Neural Information Processing Sys- 2021a.
tems,
Fang, W., Yu, Z., Chen, Y., Huang, T., Masquelier, T., and Tian, Y. Deep residual learning in spiking neural net- works. Advances in Neural Information Processing Sys- tems, 14-21056-2109 2021b.
Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
Fang, W., Yu, Z., Chen, Y., Masquelier, T., Huang, T., and Tian, Y. Incorporating learnable membrane time constant to enhance learning of spiking neural networks. In Pro- ceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2661-2671, 2021c
References
Akopyan, F. Sawada, J., Cassidy, A., Alvarez-Icaza, R.,
Arthur, J., Merolla, P., Imam, N., Nakamura, Y., Datta, P., Nam, G.-J., al. Truenorth: Design and tool flow of 65 mw million neuron programmable neurosynaptic chip. IEEE Iransactions computer-aided design of integrated circuits and systems, 34(10):1537-1557, 2015.
Feng, L.,, Liu, Q., Tang. H., Ma, D., and Pan, G. Multi-level firing with spiking ds-resnet: Enabling better and deeper directly- trained spiking neural networks. In Raedt, L. (ed.). Proceedings of the Thirty-First International Joint Conference Artificial Intelligence, IJCAI 2022, Vienna, Austria, 23-29 July 2022, pp. 2471-2477. ijcai.org, 2022. doi: 10.24963/ijcai 2022//333. URL. https: /doi. org
Bewley, A.. Ge, Z., Ott, L., Ramos, F., and Uperoft, B. Simple online and realtime tracking. In 2016 IEEE in- ternational conference on image processing (ICIP), pp. 3464 3468. IEEE, 2016.
Gong, R., Liu, X., Jiang, S., Li, T., Hu, P., Lin, J., Yu, F,, and Yan, J. Differentiable soft quantization: Bridging full- precision and low-bit neural networks. In Proceedings of the IEEE/CVE International Conference on Computer Vision, pp. 4852-4861, 2019.
Bohnstingl, T., Wožniak, S., Pantazi, A., and Eleftheriou, E. Online spatio-temporal learning in deep neural networks. IEEE Transactions on Neural Networks and Learning Systems, pp. 1-15, 2022. doi: 10.1109/TNNL. 2022. 3153985.
Guo, Y., Chen, Y., Zhang, L.. Liu, X., Wang, Y., Huang, X., and Ma, Z. IM-loss: Information maximization loss for spiking neural networks. In Oh, A. H., Agarwal, A., Belgrave, D., and Cho, K. (eds.), Advances in Neural Information Processing Systems, 2022a. URL https: / //openrevl ew net/ forum? d=Jw34v b..2b.
Davies, M., Srinivasa, N., Lin, T.-H., Chinya, G., Cao, Y., Choday, H., Dimou, G., Joshi, P.,, Imam, N., Jain, S., et al. Loihi: A neuromorphic manycore processor with on-chip learning. leee Micro, 38(1):82-99) 99, 2018
Deng, J., Dong, W., Socher, R., Li, L.JJ., Li, K., and Fei-Fei,

---

Reversing Bit of the Weight and Activation for Spiking Neural Networks
Guo, Y., Chen, Y., Zhang, L.,, Wang, Y., Liu, X., Tong. X., Ou, Y., Huang, X., and Ma, Z. Reducing information loss for spiking neural networks. In Avidan, S., Brostow, G., Cissé, M., Farinella, G. M., and Hassner, T. (eds.), Computer Vision - ECCV 2022, pp. 36-52, Cham, 2022b. Springer Nature Switzerland. ISBN 978-3-031-200821
Guo, Y., Liu, X., Chen, Y., Peng, W., Zhang, Y., and Ma, Z. Spiking transformer: Introducing accurate addition- only spiking self-attention for transformer. arXiv preprint arXiv:2503.00m 00226,, 2025.
Han, B. and Roy, K. Deep spiking neural network: En ergy efficiency through time based coding. In European Conference Computer Vision, pp. 388-404. Springer, 2020.
Guo, Y., Tong, X., Chen, Y., Zhang, L., Liu, X., Ma, Z.,, and Huang, X. Recdis-snn: Rectifying membrane po- tential distribution for directly training spiking neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp- 335, June 2022c.
Han, B., Srinivasan, G., and Roy, K. Rmp-snn: Residual membrane potential neuron for enabling deeper high- accuracy and low-latency spiking neural network. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 13558-13567, 2020.
Guo, Y., Zhang, L.,, Chen, Y., Tong, X., Liu, X., Wang, Y., Huang, X., and Ma, Z. Real spike: Learning real-valued spikes for spiking neural networks. In Computer Vision- ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XII. pp. 52-68. Springer, 2022d.
Hao, Z., Shi, X., Huang, Z., Bu, T., Yu, Z., and Huang, T. A progressive training framework for spiking neural networks with learnable Twelfth International Conference on Learning Represen- lations, 2023.
model. In The
Guo, Y., Huang, X., and Ma, Z. Direct learning- based deep spiking neural networks: a review. Frontiers in Neuroscience, 17: 17:995,,, 2023a.
He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learn- ing for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Guo, Y., Peng, W., Chen, Y.. Zhang, L., Liu, X., Huang. X., and Ma, Z. Joint a-snn: Joint training of artificial and spiking neural networks via self-distillation and weight factorization. Pattern Recognition: The Journal of the Pattern Recognition Society, 2023b.
He, Y., Zhang, X., and Sun, J. Channel pruning for acceler- ating very deep neural networks. In Proceedings of the IEEE international conference on computer vision, pp. 1389-1397, 2017.
Guo, Y., Zhang, Y., Chen, Y., Peng, W., Liu, X., Zhang, L., Huang. X., and Ma, Z. Membrane potential batch nor- malization for spiking neural networks. In Proceedings of the IEFF/CVF International Conference on Computer Vision, pp. 19420-19430, 2023c.
Hinton, G., Vinyals, O., and Dean, J. Distilling the knowledge in neural network, 2015. URL https: / //axxiv.org/// abs
Hu, Y., Tang, H., and Pan, G. Spiking deep residual net- works. IEEE Transactions on Neural Networks and Learning Systems, 2021.
Guo, Y., Chen, Y., Hao, Z., Peng, W., Jie, Z.,, Zhang, Y., Liu, X., and Ma, Z. Take shortcut back: Mitigating the gradient vanishing for training spiking neural networks. arXiv preprint arXiv:2401. 2401044000. 2024a.
Huang, G., Liu, Z., van der Maaten, L., and Weinberger, K. Q. Densely connected convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.
Guo, Y., Chen, Y., Liu, X., Peng, W., Zhang, Y., Huang. X., and Ma, Z. Ternary spike: Learning temary spikes for spiking neural networks. In Proceedings of the AAAI Con- ference Artificial Intelligence, volume 38, pp. 12244- 12252, 2024b.
Jiang, H., Zoonekynd, V., De Masi, G., Gu, B., and Xiong, H. Tab: Temporal accumulated batch normalization in spiking neural networks. In The Twelfth International Conference on Learning Representations.
Guo, Y., Peng, W., Chen, Y., Zhou, J., and Ma, Z. Improved event-based image de-occlusion. IEEE Signal Processing Letters, 2024c.
Kim, S., Park, S., Na, B., and Yoon, S. Spiking-yola spik- ing neural network for energy-efficeen object detection. In Proceedings of the AAAI conference on artificial intel- ligence, volume 34, pp. 1 11270-11271 2020.
Guo, Y., Peng, W., Liu, X., Chen, Y., Zhang, Y., Tong, X., Jie, Z., and Ma, Z. Enof-snn: Training accurate spiking neural networks via enhancing the output feature. Advances in Neural Information Processing Systems, 37: 1178-51726, 2024d.
Krizhevsky, A., Nair, V., and Hinton, G. Cifar-10 (canadian institute for advanced research). URL http:///ww. cs. toronto. edufkriz/ciaa himl, 5(4):1, 2010.
10

---

Reversing Bit of the Weight and Activation for Spiking Neural Networks
Li, H., Liu, H., Ji, X., Li, G., and Shi, L. Cifar10-dvs: an event-stream dataset for object classification. Frontiers in neuroscience, 11:309, 2017.
Park, S., Kim, S., Na, B., and Yoon, S. T2fsnn: Deep spiking neural networks with time-to-fisttspikke coding. In 2020 57th ACM/IEEE Design Automation Conference (DAC), pp. 1-6. IEEE, 2020.
Li, Y., Deng, S., Dong, X., Gong, R., and Gu, S. A free lunch from ann: Towards efficient, accurate spiking neu- ral networks calibration. In International Conference on Machine Learning, pp. 6316-6325, PMLR, 2021a.
Pei, J., Deng, L., Song, S., Zhao, M., Zhang. Y., Wu, S., Wang, G., Zou, Z.., Wu, Z.,, He, W., et al. Towards artificial general intelligence with hybrid tianjic chip architecture. Nature, 372(7767):106- 111, 2019.
Li, Y., Guo, Y., Zhang. S., Deng, S., Hai, Y., and Gu, S. Dif- ferentiable spike: Rethinking gradient-descent for train- ing spiking neural networks. Advances in Neural Infor- mation Processing Systems, 34:23426-23439, 2021b.
Polino, A., Pascanu, R., and Alistarh, Model compres- sion via distillation and quantization. arXiv preprint arXiv: arXiv: 0et.10002..0.66. 2018.
Qin, H., Zhang, Y., Ding, Y., Liu, X., Danelljan, M., Yu, F., et al. Quantsr: accurate low-bit quantization for efficient image super-resolution. Advances in Neural Information Processing Systems, 36, 2024.
Liu, F., Zhao, W., Chen, Y., Wang. Z., and Jiang, L. Spike- converter: An efficient conversion framework zipping the gap between artificial neural networks and spiking neural networks. In Thirty-Sixth AAAI Conference on Artificial Intelligence, AAAI 2022, Thirty- Confer- ence on Innovative Applications of Artificial Intelligence, IAAI 2022, The Twelveth Symposium on Educational Advances in Artificial Intelligence, EAAI 2022 Virtual Event, February 22 MMrrh 1, 2022, pp. 1692-1701. AAAI Press, 2022. URL
Rathi, N. and Roy, K. Diet-snn: Direct input encoding with leakage and threshold optimization in deep spiking neural networks. arXiv preprint arXiv:2008.0366
2020.
Rathi, N., Srinivasan, G., Panda, P., and Roy, K. Enabling
deep spiking neural networks with hybrid conversion and spike timing dependent backpropagation. arXiv preprint arXiv:2005.00 00000, 2020.
index. php/ AAAI http///articceeeeee//////
Ma, D., Shen, J., Gu, Z., Zhang, M., Zhu, X., Xu, X. Xu, Q., Shen, Y., and Pan, G. Darwin: A neuromorphic hardware co-processor based on spiking neural networks. Journal of Systems Architecture, 77:43-51, 2017.
Ren, D., Ma, Z., Chen, Y., Peng, W., Liu, X., Zhang, Y., and Guo, Y. Spiking pointnet: Spiking neural networks for point clouds. arXiv preprint arXiv:2310.06232 2023.
Maass, W. Networks of spiking neurons: The third generation of neural network models. Neural Net- works, 16991116,,,, 1997. ISSN 0893-6080.
Ronneberger, O., Fischer, P., and Brox, T. U-net: Convolu- tional networks for biomedical image segmentation. In In- ternational Conference on Medical image computing and computer-assisted intervention, pp. 234-241. Springer, 2015.
doi: http://kciaogg///000000ss3320000
URL htttpss//www.sccieneeddreee http:///actcceee///////311660110
Sengupta, A., Ye, Y., Wang, R., Liu, C., and Roy, K. Going deeper in spiking neural networks: Vgg and residual architectures. Frontiers in neuroscience, 13:95, 2019.
Meng, Q.. Xiao, M., Yan, S.. Wang, Y., Lin, Z., and Luo, Z.- Q Training high-performance low-latency spiking neural networks by differentiation on spike representation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22444-1255. 2022.
Subramoney, A., Nazeer, K. K., Schöne, M., Mayr, C., and Kappel, D. Efficient recurrent architectures through ac- tivity sparsity and sparse pack-propagation through time. In The Eleventh International Conference on Learning Representations, 2023. URL ht opwnreviee...... net mor/fo http:/oorumidd111d00a 1eee:dddd11doooo 1d=11do11aaaadd.
Ming, Q. Miao, L.,, Ma, 7.,, Zhao, L.., Zhou, Z., Huang, X., Chen, Y., and Guo, Y. Deep dive into gradients: Better optimization for 3d object detection with gradient- corrected iou supervision. In 2023 IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 5136-5145, 2023. doi: 10. 10.1109/CVPRS2279.200 00497.
Sun, C., Chen, Q., Fu, Y., and Li, L. Deep spiking neural network with ternary spikes. In 2022 IEEE Biomedical Circuits and Systems Conference (BioCAS), pp. 251-254. IEEE, 2022.
Neftci, E. O., Mostafa, H., and Zenke, F. Surrogate gradient learning in spiking neural networks: Bringing the power of gradient- baadd optimization to spiking neural networks. IEEE Signal Processing Magazine, 36(6):51 2019.
Wang, S., Schmutz, V., Bellec, G., and Gerstner, W. Meso- scopic modeling of hidden spiking neurons. In Oh, A. H., Agarwal, A., Belgrave, D., and Cho, K. (eds.), Advances in Neural Information Processing Systems,

---

Reversing Bit of the Weight and Activation for Spiking Neural Networks
2022. URL httph::://pperreeiieeeeee// id=cyPja_wjdd.
Yao, X., Li, F,, Mo, Z., and Cheng, J. Glif: A unified gated leaky integrate-and-fiee neuron for spiking neural networks. arXiv preprint arXiv:2210.13700 2022.
Wang, X., Zhang, Y., and Zhang, Y. MI-sn: Enhance spiking neural network with multiple thresholds, 2023.
Yu, K., Yu, C., Zhang, T., Zhao, X., Yang, S., Wang, H., Zhang, Q. and Xu, Q. Temporal separation with entropy regularization for knowledge distillation in spiking neural networks. arXiv preprint arXiv:2503.00 50330034.44, 2025.
Wu, J., Chua, Y. Zhang, M., Li, G., Li, H., and Tan, K. C. A tandem learning rule for effective training and rapid infer- ence of deep spiking neural networks. IEEE Transactions on Neural Networks and Learning Systems, 2021a.
Yu, Q., Ma, C., Song, S., Zhang, G., and Tan, structing accurate and efficient deep spiking neural net- works with double-threshold and augmented schemes. IEEE Transactions on Neural Networks and Learning Systems, 1P(99):1-13,2000
Con-
Wu, J., Chua, Y., Zhang, M., Li, G., Li, H., and Tan, K. C. tandem learning rule for effective training and rapid infer- ence of deep spiking neural networks. IEEE Transactions on Neural Networks and Learning Systems, 2021b.
Zhang. L., Bao, C., and Ma, K. Self-distillation: Towards efficient and compact neural networks. IEEE Transac- tions Pattern Analysis and Machine Intelligence, 44(8): 1388-4407 2022. doi: 10.1 109/TPAMM.2221.3322 20113306000000.0061000
Wu, J., Xu, C., Han, X., Zhou, D., Zhang, M., Li, H., and Tan, K. C. Progressive tandem learning for pattern recog- nition with deep spiking neural networks. IEEE Transac- tions on Pattern Analysis and Machine Intelligence, 44 (11):7824-7840, 2021c
Zhang, W. and Li, P. Temporal spike sequence learning via backpropagation for deep spiking neural networks. Advances in Neural Information Processing Systems, 33: 12022-12033, 2020.
Wu, Y., Deng, L., Li, G.. Zhu. J., and Shi, L. Spatio- temporal backpropagation for training nigh-performance spiking neural networks. Frontiers in neuroscience, 12: 331, 2018
Zhang. Y., Liu, X., Chen, Y., Peng, W., Guo, Y., Huang, X., and Ma, Z. Enhancing representation of spiking neural networks via iimilarity-sensitivv contrastive learning. In Proceedings of the AAAI Conference Artificial Intelli- gence, volume 38, pp. 16926 16934, 2024.
Wu, Y., Deng, L., Li, G., Zhu, Xie, Y., and Shi, L. Direct training for spiking neural networks: Faster, larger, bet- ter. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp. 1311-1318, 2019.
Zheng, H., Wu, Y., Deng, L., Hu, Y., and Li, G. Going deeper with directly-trained larger spiking neural net- works. In Proceedings of the AAAI Conference on Artifi- cial Intelligence, volume 35, pp. 11062-11070 2021.
Xiao, M., Meng, Q. Zhang, Z., Wang, Y., and Lin, Z. Train- ing feedback spiking neural networks by implicit differ- entiation on the equilibrium state. Advances in Neural Information Processing Systems, 34:14516 145228, 2021.
Zhou, Z., Zhu, Y., He, C., Wang, Y., YAN, S., Tian, Y., and Yuan, L. Spikformer: When spiking neural network meets transformer. In The Eleventh International Conference on Learning Representations, 2023. URL https:/ openreview net for
Xiao, M., Meng, Q., Zhang, Z., He, D., and Lin, Z. On- line training through time for spiking neural networks. In Oh, A. H., Agarwal, A., Belgrave, D., and Cho, K. (eds.), Advances in Neural Information Processing Systems, 2022. URL https: / www.review.ww....///p/.. / forum? 33-Sii3nHHhhe......
Xu, Q., Li, Y., Shen, J., Liu, J. K., Tang, H., and Pan, G. Constructing deep spiking neural networks from artifi- cial neural networks with knowledge distillation. arXiv preprint ar 11:22.304006677, 2023.
Yang, Q. Wu, J., Zhang, M., Chua, Y., Wang, X., and Li, H. Training spiking neural networks with local tandem learning. arXiv preprint arXiv:2210.04532. 2022.
Yao, M., Hu, J., Zhou, Z., Yuan, L., Tian, Y., Xu, B., and Li, G. Spike- transformer. Advances in neural information processing systems, 001644433600555 2023.
12


Reviews:
[{"review_id": "w1oNLHVuVL", "reviewer": "Reviewer_LZsQ", "summary": "In this paper, the authors propose to make the weights of a spiking neural network ternary and the spikes of the units in the network real valued, essentially swapping what is done in SNNs usually. This swap preserves the advantages of SNNs, while improving its expressivity. The authors demonstrate their method in a number of benchmarks.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your efforts in reviewing our paper and your recognition of our novel method and notable results. The responses to your concerns and questions are given piece by piece as follows.\n\n**Concern 1**: It would have been useful to also see evaluations on sequence tasks since SNNs are inherently recurrent.\n\n**R1**: Thanks for the advice. Our method also performs well in sequence tasks. We have added the result on the CIFAR10-DVS and DVS-Gesture as below. It can be seen that our method also performs on par with or better than state-of-the-art methods.\n\n| Dateset | Method | Architecture | Timestep | Accuracy |\n| --- | --- | --- | --- | --- |\n| CIFAR10-DVS | SSCL(AAAI 2024) | ResNet19 | 10 | 80.00% |\n| CIFAR10-DVS | SpikeFormer(ICLR 2023) | SpikeFormer | 10 | 78.90% |\n| CIFAR10-DVS | **Our method** | ResNet19 | 10 | **80.50%** |\n| DVS-Gesture | ASA-SNN(ICCV 2023) | 5 layer SCNN | 20 | 97.70% |\n| DVS-Gesture | SpikeFormer(ICLR 2023) | SpikeFormer | 20 | 96.90% |\n| DVS-Gesture | TCJA (TNNLS 2024) | 5 layer SCNN | 20 | 97.56% |\n| DVS-Gesture | **Our method** | 5 layer SCNN | 20 | **98.23%** |\n\n**Concern 2**: Why do the authors choose to use STBP to train the network rather than standard BPTT?\n\n**R2**: Sorry for the confusion.  Considering the similarity in computational mechanisms between SNNs and Recurrent Neural Networks (RNNs), SNN researchers transferred the Back-propagation Through Time (BPTT) method from RNNs to the supervised learning field of SNNs, which is also called the STBP training algorithm. Thus BPTT is the same as STPB in SNNs.\n\n**Concern 3**: One minor weakness is the framing of the method within the current literature.\n\n**R3**: Thanks for the advice. We will add more related literature and reframe them in the final version, such as:\n\n[1] Subramoney, A., Nazeer, K. K., Schöne, M., Mayr, C. & Kappel, D. Efficient recurrent architectures through activity sparsity and sparse back-propagation through time. in The Eleventh International Conference on Learning Representations (2023).\n\n[2] Maass, W. Networks of spiking neurons: The third generation of neural network models. Neural Networks 10, 1659–1671 (1997).\n\n**Concern 4**: It's not clearly described if a value of $\\alpha$ is shared across an entire layer.\n\n**R4**: Sorry for the confusion.  $\\alpha$ is not shared across an entire layer. It is applied in a channel-wise manner across our models. We will make this description clearer in the final version.\n\n**Questions  1**: What is the activation used in between layers?\n\n**A1**: Sorry for the confusion. In addition inherently recurrent of our activation, the output of our activation is a ReLU-like function defined as follows:\n$ {\\rm O}= {\\rm U} \\\\  {\\rm if U} \\\\ \\ge V_{\\rm th}, \\\\ { \\rm otherwise} \\\\ 0$\n\nIn Relu activation, the $V_{\\rm th}$ is 0, while in our activation, the $V_{\\rm th}$ can be adjusted across different datasets or scenes. Our activation will decay through time, while ReLU not.", "claims_and_evidence": "The claims made in the paper are supported by clear and convincing evidence -- both for their method preserving the advantages of SNNs in terms of energy efficiency (Table 5) and its ability to outperform existing SNNs in various benchmarks (Table 2-4).", "methods_and_evaluation_criteria": "The benchmarks in which the method is evaluated on are pretty standard in the field for feed-forward SNNs. But it would have been useful to also see evaluations on sequence tasks since SNNs are inherently recurrent.\n\nIt is not completely clear why the authors choose to use STBP to train the network rather than standard BPTT. This decision could be explained a bit better.", "theoretical_claims": "No major theoretical claims.", "experimental_designs_or_analyses": "The ablation studies, evaluation on standard benchmarks, and energy efficiency are the analyses that the authors perform. Did not find any major issues with any of them.", "relation_to_broader_scientific_literature": "This paper builds on existing SNNs, improving their expressivity while keeping their other advantages. The use of ternary weights and real valued spikes [1] have been done before, but not in this specific context.\n\n[1] Subramoney, A., Nazeer, K. K., Schöne, M., Mayr, C. & Kappel, D. Efficient recurrent architectures through activity sparsity and sparse back-propagation through time. in The Eleventh International Conference on Learning Representations (2023).", "essential_references_not_discussed": "The use of real-valued activations in spiking (i.e. event-based) networks has been done before in [1], and should be discussed. But the specific combination of ternary weights and real-valued activation is novel to my knowledge.\n\n[1] Subramoney, A., Nazeer, K. K., Schöne, M., Mayr, C. & Kappel, D. Efficient recurrent architectures through activity sparsity and sparse back-propagation through time. in The Eleventh International Conference on Learning Representations (2023).", "other_strengths_and_weaknesses": "The key concept in this paper is novel to my knowledge. The end to end training setup and evaluation on challenging benchmarks is also a strength.\n\nOne minor weakness is the framing of the method within current literature, which is not done thoroughly. For example many of the papers referenced in the introduction seem very arbitrary or is missing references to the seminal papers. E.g. The usual reference used for SNNs is [1]. Quantization was known well before 2019. Ditto knowledge distillation etc.\n\n[1] Maass, W. Networks of spiking neurons: The third generation of neural network models. Neural Networks 10, 1659–1671 (1997).", "other_comments_or_suggestions": "- It's not clearly described if a value of $\\alpha$ is shared across an entire layer.", "questions_for_authors": "- What is the activation used in between layers? In Fig. 1, it looks like it uses ReLU activations?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wXPfLP4DQM", "reviewer": "Reviewer_jZSA", "summary": "This paper addresses the issue of information loss in Spiking Neural Networks (SNNs) due to the binarization of activations. The main contribution in the paper is to use binary weights (in $\\\\{-1,1\\\\}$) and real-valued spikes, instead of binary spikes and real-valued weights. This initial contribution is extended by allowing the binary weights to take their values in $\\\\{-\\alpha, \\alpha\\\\}$, where $\\alpha$ is a trainable parameter; then, a re-parametrization technique is proposed to take $\\alpha$ into account at inference while going back to binary weights in $\\\\{-1,1\\\\}$. Experiments on three datasets (CIFAR-10, CIFAR-100, ImageNet, with two architectures tested per dataset) are reported. Results suggest that the proposed contributions improve the classification accuracy over the baseline architectures, and perform on par with or better than state-of-the-art architectures.\n\n## Update after rebuttal\nThe rebuttal of the authors addressed my concerns about the rationale and design of the contribution. However, I still have concerns about the way hyperparameters were chosen/optimized, and the validity of the model used to estimate energy consumption. So, I increase my score to _2. Weak reject_.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your efforts in reviewing our paper. We will try to make the work clearer for you. The responses to your concerns and questions are given as follows.\n\n**Concern 1**: The only required addition is not for the learnable version.\n\n**R1**: Sorry for the confusion. Since the $\\alpha$ will be fixed after training, it can be folded to the activation function before inference. Then only requires additions too.\n\n**Concern 2**: the model may not have the low power consumption benefits. \n\n**R2**: Thanks for the question. I agree that on many low-power neuromorphic hardware, the low-power consumption is thanks to the use of binary spikes. However, there are also many hardware that realize low-power consumption based on replacing multiplications with additions like [1,2]. With this hardware, our model can keep low power consumption too. What’s more, there is also no available hardware that could support SNN-based transformer architectures which become popular now. Our model and SNN-based transformer architectures could drive further hardware development like the emergence of other hardware like Lohoi and TureNorth. \n\n[1] A systolic SNN inference accelerator and its co-optimized software framework.\n\n[2] Tianjin chip. \n\n**Concern 3**: The evaluation of memory consumption.\n\n**R3**: Thanks for the question. For memory consumption, the vanilla SNN’s weights adopt 32 bits while our model’s weights could adopt 1 bit. Thus the memory consumption of our model is much less than that of the vanilla SNN.\n\n**Concern 4**: Provide the derivation for Equations 11, 14, and 15.\n\n**R4**: Thanks for the question. For Equations 11, from Equation 5 and $\\mathbf{W_l}^b = {\\rm sign}(\\mathbf{W_l})$, based on chain rule, we could know that $\\frac{\\partial {L}}{\\partial {\\mathbf{W}_l}} = \\sum_t \\frac{\\partial {L}}{\\partial {U^t_l}}\\frac{\\partial {U^t_l}}{\\partial {\\mathbf{W}^b_l}}\\frac{\\partial \\mathbf{W}^b_l}{\\partial {\\mathbf{W}_l}}$. From Equation 4 and Equation 1, we know that $U^t_l$ will affect $O^t_l$ and $U^{t+1}_l$, thus $\\frac{\\partial {L}}{\\partial {U^t_l}} = \\frac{\\partial {L}}{\\partial {O^t_l}} \\frac{\\partial {{O^t_l}}}{\\partial {{U^t_l}}} +  \\frac{\\partial {L}}{\\partial {{{U^{t+1}_l}}}} \\frac{\\partial {{U^{t+1}_l}}}{\\partial {{U^t_l}}}$. Combine all these, we can get Equations 11. \n\nThe proof of Equation 14 is the same as Equation 11. \n\nFor Equation  15, the $\\alpha$ only affects $U^t$, thus based on the derivation of Equation 11, it is easy to derive it.\n\n**Concern 5**: Equations 14 and 11 are the same?\n\n**R5**: Sorry for the confusion.  $\\mathbf{W}^b_{\\rm trainble} = \\alpha \\mathbf{W}^b$, and we calculate $\\alpha$ and $W$ separately in Equations 14 and 15. Thus Equations 14 and 11 are the same in form.\n\n**Concern 6**: The firing activity becomes differentiable\" should be further justified. \n\n**R6**: Thanks for the advice. Compared to binary activation, our real-valued activation becomes differentiable in more intervals. We overclaimed this in the paper. We will correct this in the final version. Thanks.\n\n**Concern 7**: The firing threshold is said to be set to 0?\n\n**R7**: Sorry for the confusion. In these static datasets, it is 0 all the time, since static datasets can not provide timing information. For neuromorphic datasets, we set it to 0.25.\n\n**Concern 8**: the experimental settings are missing.\n\n**R8**: Thanks for the question. We will clarify the code settings in detail in the final version.\n\n**Concern 9**: In Table 5, the figures for learnable ReverB are missing.\n\n**R9**: Thanks for the question.  We add the results for learnable ReverB below. \n\n| Accuracy | #Flops |  #Sops | Energy |\n| --- | --- | --- | --- |\n| 94.45% | 3.54M |  75.10M | 50.03uJ |\n\n**Concern 10**: Why $\\alpha \\in \\mathbb{R}^{C \\times 1 \\times 1}$\n\n**R10**: Sorry for the confusion. $\\alpha$ is in a channel-wise manner for convolution layers. \n\n**Concern 11**: Some elements in Algorithm 1 are not clear.\n\n**R11**: Sorry for the confusion. The `for` loop is over mini-batches. With the $\\alpha$ folded to i-1 function, then we can obtain a standard ReverB-SNN. We will correct the Inference to Evaluation in the Algorithm. Thanks. \n\n**Concern 12**: Other Comments Or Suggestions.\n\n**R12**: Very much thanks for these kind reminders. We will carefully correct these in our final version.\n\n**Question 1**: Why considering binary weights not ternary weights.\n\n**A1**: Thanks for the question. Using ternary weights is better than binary weights in our experiments. However, we only report the binary weight and real-valued activate results for a fair comparison of vanilla real-valued weight and binary activate SNNs.\n\n**Question 2**: How to deploy the proposed model to neuromorphic hardware.\n\n**A2**: Thanks for the question. Please see our response for **Concern 2**.", "claims_and_evidence": "The paper makes the following claims:\n1. Weight binarization is less detrimental to performance than activation binarization. This statement seems to be supported by experimental results.\n2. The proposed method (binarized weights, non-binarized spikes) maintains the event-based nature of the network. This claim seems correct, by construction.\n3. The proposed method only requires additions. It seems to only apply to inference, not training, although this is no explicitly stated in the paper. While this claim is true for the first version of ReverB, it looks like this is not the case for the learnable version, which requires multiplications, according to Equation 18.", "methods_and_evaluation_criteria": "1. The paper proposes to solve the issue of information loss due to binary spikes in SNNs by binarizing weights instead of spikes. On the one hand, I find it interesting to change the perspective on this issue by moving the quantization issue from one variable of the model to another. On the other hand, I wonder whether this approach could be applied in practice. SNNs are valuable when they can be implemented on low-power neuromorphic hardware. Low power consumption is made possible thanks to the use of binary spikes by the model, and neuromorphic hardware is designed with this in mind. Switching binarization from activations to weights may not have the same benefits. This question is not addressed in the paper.\n\n2. The evaluation criteria are the accuracy of the model and its energy consumption, which are relevant criteria in this context. However, the evaluation of the energy consumption of the model is based on the method from (Hu et al., 2021), who base their estimation on the specifications of particular hardware devices. These devices may not be able to run natively networks with real-valued spikes like the ones proposed in this paper. The relevance of this methodology for the network described here is not demonstrated in the paper. In addition, I believe memory consumption should also be considered as an evaluation criteria, as, here again, it may be very different with the proposed model.", "theoretical_claims": "1. The paper provides the formulas for gradient computation in the proposed models (Equations 11, 14, and 15), but do not provide the derivation of these results, so I could not check whether they are sound.\nThe authors should provide (for instance, as supplementary material) the complete demonstration that leads to these equations, as it is not straightforward considering the changes they made to the initial model (continuous weights, binary spikes) used in the STBP paper (Wu et al, 2018).\n\n2. Equations 14 and 11 are the same, whereas Equation 14 should show how to compute the gradients with $\\mathbf{W}^b_{\\mathrm{trainable}}$. I believe there might be an error here.\n\n3. The authors state (Section 3.2) that \"the firing activity of spiking neurons becomes differentiable\" in their model. This statement should be further justified. To my understanding, although the spikes take real values, they are still local in time and generated through a thresholding function, so there are discontinuities in the activation function that should be problematic in terms of differentiability.", "experimental_designs_or_analyses": "I reviewed Section 4 entirely. My comments about the experiments are detailed below.\n\n1. The firing threshold $V_{\\mathrm{th}}$ is said to be set initially to 0. Is this the actual value or a typo? Also, can it change over time? This was not mentioned previously in the paper.\n\n2. A number of elements are missing from the experimental settings, which prevents from reproducing the experiments:\n    - the optimizer used to train the network and its hyperparameters (e.g., number of epochs),\n    - the protocol used to determine hyperparameters,\n    - data pre-processing (if any),\n    - preparation of the data (train/validation/test splits, and the size of mini-batches).\n\n3. In Table 5, the figures for learnable ReverB are missing.", "relation_to_broader_scientific_literature": "1. The problem addressed in this paper is relevant to the community of neuromorphic machine learning. It has been addressed in a number of previous papers, as mentioned in Section 2. To the best of my knowledge, there is no previous work that proposes the same contribution as the one in this paper.\n\n2. The use of binary weights has been explored in standard ANNs, for instance in (Courbariaux et al., 2015). The paper does not mention this line of work.\n\n- (Courbariaux et al., 2015) M. Courbariaux, Y. Bengio, J.P. David. BinaryConnect: Training Deep Neural Networks with binary weights during propagations. Neural Information Processing Systems (NeurIPS) (2015).", "essential_references_not_discussed": "I did not identify any essential references that are not cited.", "other_strengths_and_weaknesses": "1. After Equation 12, it is stated that $\\alpha \\in \\mathbb{R}^{C \\times 1 \\times 1}$. This dimension is not quite clear to me, since I assumed from the beginning that the previous equations were about fully-connected layers. It seems that the dimensions of tensors do not match here. The dimensions of tensors should be provided to make this clearer.\n\n2. Some elements in Algorithm 1 are not clear.\n   - It is not clear whether the `for` loop in the training algorithm loops over mini-batches or epochs.\n   - In line 2 of the re-parameterization algorithm, why does $\\alpha_i$ unfold into $i-1$ functions?\n   - Why are labels used for inference? They should be used for evaluation only, not inference.", "other_comments_or_suggestions": "1. Figure 1 is not very informative, as the principle of the contribution is simple enough and clearly stated in the paper. This figure could be removed.\n\n2. In Equation 3, $T$ is not defined.\n\n3. (Rathi & Roy, 2020) is cited to motivate the choice of the surrogate gradient, however a different surrogate gradient function is used in that paper.\n\n4. In Section 3.2, paragraphs \"Event-based Advantage Retaining\" and \"Addition-only Advantage Retaining\" could be significantly shortened as they state straightforward properties of their model.\n\n5. Parentheses are not typeset correctly in several equations (`\\left(` and `\\right)` should be used).\n\n6. Before Equation 16, I think the equation number in \"Eq. 16 can be further written as\" is not the right one.\n\n7. In Section 4.2, it is stated that top-1 accuracy and the mean accuracy and standard deviation are presented. I guess what is meant is that mean (top-1) accuracy is reported?\n\n8. Percentages (%) are used instead of percentage points (pp) when presenting differences in accuracies.\n\n9. Some typos should be corrected:\n    - \"trainable\" is misspelled \"trainble\" in equations,\n    - \"binarization\" is misspelled \"binaration\" after Equation 11,\n    - \"will converted\": \"will be converted\",\n    - before Equation 17, I guess $\\alpha_1$ should actually be $\\alpha_l$,\n    - in the description of the experimental settings (Section 4), $\\tau$ becomes $\\tau_{\\mathrm{decay}}$,\n    - \"peak accuracies of 95.51%\": \"peak accuracies of 95.53%\",\n    - \"does not enjoy the multiplication-free\" needs to be rephrased (Section 5),\n    - \"obviously\" -> \"Obviously\",\n    - the title of Section 4 should be spelled \"Experiments\".\n\n10. The paper contains some subjective over-statements, which should be avoided. For instance:\n    - \"the well-trained SNN\" (in Algorithm 1),\n    - \"Remarkably, our method achieves [...]\",\n    - \"achieved impressive accuracies\".", "questions_for_authors": "1. Why considering binary weight in $\\\\{-1,1\\\\}$ and not ternary weights in $\\\\{-1,0,1\\\\}$? The latter could enhance the learning capacity of neurons.\n\n2. SNNs are valuable when they can be deployed on low-power neuromorphic hardware. Such hardware is typically designed with binary spikes in mind. Can the authors elaborate on the compatibility of their approach with current neuromorphic hardware, and what changes (if needed) should be applied to neuromorphic architectures to run this type of model?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KhTRC7vLnM", "reviewer": "Reviewer_LvHJ", "summary": "The paper proposes an SNN design with real-valued activations and binary weights to boost information capacity while keeping energy efficiency. Its novel bit-reversal strategy and adaptive weight scaling are key innovations. However, the paper’s motivation and presentation lack clarity and could benefit from visual aids.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your efforts in reviewing our paper and your recognition of our novel bit-reversal strategy and adaptive weight scaling. The responses to your weaknesses and questions are given piece by piece as follows.\n\n**Weakness 1**: The paper does not clearly explain the rationale behind reversing the bits of weight and activation. This lack of clarity could make it challenging for readers to fully grasp why this approach is beneficial.\n\n**R1**: Thanks for the advice. Here, we will use the entropy concept to show that ReverB-SNN has a higher information capacity than vanilla SNN.\n\nwe first perform a theoretical analysis using the concept of entropy. The representational capability $ \\mathcal{C}(\\mathbf{X}) $ of a set $ \\mathbf{X} $ is determined by the maximum entropy of $ \\mathbf{X} $, expressed as:\n$\\mathcal{C}(\\mathbf{X}) = \\max \\mathcal{H}(\\mathbf{X}) = - \\sum_{x \\in \\mathbf{X}} p_{\\mathbf{X}}(x) \\log p_{\\mathbf{X}}(x),$ where $ p_{\\mathbf{X}}(x) $ is the probability of a sample $ x $ from $ \\mathbf{X} $. Then it is clear for the following proposition:\n\nFor a set $ \\mathbf{X} $, when the probability distribution of $ \\mathbf{X} $ is uniform, i.e., $ p_{\\mathbf{X}}(x) = \\frac{1}{N} $, where $ N $ is the total number of samples in $ \\mathbf{X} $, the entropy $ \\mathcal{H}(\\mathbf{X}) $ reaches its maximum value of $ \\log(N) $. Hence, we conclude that $ \\mathcal{C}(\\mathbf{X}) = \\log(N) $.\n\nUsing the proposition, we can evaluate the representational capacity of vanilla SNN and our model.\n\nLet's consider two connected neuron layers. Since the connectivity of the neurons is the same no matter for vanilla SNN or our model, let us focus on an arbitrary two connected neurons from different layers. \n\nFor the vanilla SNN, the information output from one neuron to another are {0, 1} x W, where W is the fixed weight between the two neurons. Thus one neuron from vanilla SNN could transform two samples into another one, and $ \\mathcal{C}(\\mathbf{X}) = \\log(2) = 1$. \n\nFor our model, the information output from one neuron to another are o x {-1, 1}, where o is the real-valued spike and could be changed. o requires 32-bits. Thus the number of possible samples from o is 2^{32} and the value samples from one neuron to another is 2^{32+1}. $ \\mathcal{C}(\\mathbf{X}) = \\log(2^{32+1}) = 33$.\n\nThis highlights the limited representational capacity of vanilla SNN compared to our model.\n\n**Weakness 2**: The section detailing the contributions is somewhat lengthy and could be streamlined. Consolidating similar ideas might improve readability and focus.\n\n**R2**: Thanks for the advice. We will further polish our contributions in the final version.\n\n**Weakness 3**: The comparisons in Table 2 and Table 4 appear to rely on methods from 2022, missing more recent literature.\n\n**R3**: Thanks for the advice. We have added more comparisons as below. It can be seen that our method also performs on par with or better than state-of-the-art methods.\n\n| Dateset | Method | Architecture | Timestep | Accuracy |\n| --- | --- | --- | --- | --- |\n| CIFAR10 | Q-SNNs(ACMMM 2024) | ResNet19 | 2 | 95.54% |\n|  | AGMM(AAAI 2025) | ResNet19 | 2 | 96.33% |\n|  | FSTA-SNN(AAAI 2025) | ResNet20 | 4 | 94.72% |\n|  | FSTA-SNN(AAAI 2025) | ResNet19 | 2 | 96.52% |\n|  | TAB(NeurIPS 2024) | ResNet19 | 2 | 94.73% |\n|  | **Our method** | ResNet20 | 4 | **94.96%** |\n|  | **Our method** | ResNet19 | 2 | **96.62%** |\n| CIFAR100 | SSCL(AAAI 2024) | ResNet20 | 2 | 72.86% |\n|  | SSCL(AAAI 2024) | ResNet19 | 2 | **78.79%** |\n|  | TAB(NeurIPS 2024) | ResNet19 | 2 | 76.31% |\n|  | **Our method** | ResNet20 | 4 | **73.28%** |\n|  | **Our method** | ResNet19 | 2 | 78.46% |\n| CIFAR10-DVS | SSCL(AAAI 2024) | ResNet19 | 10 | 80.00% |\n|  | SpikeFormer(ICLR 2023) | SpikeFormer | 10 | 78.90% |\n|  | **Our method** | ResNet19 | 10 | **80.50%** |\n\n**Weakness 4**: The paper could benefit from additional visualizations. Graphs or schematic diagrams illustrating the architecture and the re-parameterization process would enhance understanding and provide clearer insights into the proposed method.\n\n**R4**: Thanks for the advice. We have added the visualizations for the re-parameterization process. Please see it from https://imgur.com/GiJYTke\n\n**Question 1**: Could you clarify the theoretical motivation and advantages behind \"Reversing Bit\"?\n\n**A1**: Thanks for the question.  Please see our response to **Weakness 1.**\n\n**Question 2**: The paper lacks visualizations; could you add diagrams of the network architecture or re-parameterization process for clarity?\n\n**A1**: Thanks for the advice. We have added the visualizations for the re-parameterization process. Please see it from https://imgur.com/GiJYTke", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "The paper doesn't provide formal proofs but offers partial derivations that seem generally sound, though some steps lack full justification, leaving a degree of uncertainty.", "experimental_designs_or_analyses": "The overall experimental design appears sound and appropriate for the problem at hand. While some aspects could be clarified further, these do not undermine the main findings.", "relation_to_broader_scientific_literature": "The paper builds on established SNN research, particularly findings that quantizing activations causes more accuracy loss than weights. Its contributions extend prior work on efficient, multiplication-free SNNs and adaptive parameterization.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "Strengths\n+ The approach preserves the multiplication-free, event-driven nature of SNNs, ensuring that energy efficiency remains a strong point despite the introduction of real-valued activations.\n+ Incorporating a trainable factor for binary weights and using re-parameterization during inference allows the network to learn optimal weight magnitudes while still converting to a standard binary format. This offers a good balance between learning flexibility and inference efficiency.\n\nWeaknesses\n- The paper does not clearly explain the rationale behind reversing the bits of weight and activation. This lack of clarity could make it challenging for readers to fully grasp why this approach is beneficial.\n- The section detailing the contributions is somewhat lengthy and could be streamlined. Consolidating similar ideas might improve readability and focus.\n- The comparisons in Table 2 and Table 4 appear to rely on methods from 2022, missing more recent literature. \n- The paper could benefit from additional visualizations. Graphs or schematic diagrams illustrating the architecture and the re-parameterization process would enhance understanding and provide clearer insights into the proposed method.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "1. Could you clarify the theoretical motivation and advantages behind \"Reversing Bit\"?\n2. The paper lacks visualizations; could you add diagrams of the network architecture or re-parameterization process for clarity?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PnDn3TSaHZ", "reviewer": "Reviewer_aSHw", "summary": "This paper introduces a novel binary design in SNN termed ReverB, which uses real-value activation and binary weights, merging the characteristics of both BNN and SNN. This innovative approach retains the energy efficiency advantages of SNNs in inference and their temporal properties. Additionally, the paper proposes a reparameterization method to enhance the performance of the ReverB network.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your efforts in reviewing our paper and your recognition of our novel method and effective results. The responses to your questions are given piece by piece as follows.\n\n**Question 1**: Does the ReverB network has a reset mechanism, and how does it function?\n\n**A1**: Thanks for the question. ReverB network has a reset mechanism. When the membrane potential exceeds the firing threshold, it will emit itself and then reset to 0; otherwise, it will decay to a new value and then be updated with next-time input.\n\n**Question 2**: Does the ReverB network use the Batch Normalization (BN) layer, and can ReverB eliminate it during inference similar to SNN? \n\n**A2**: Thanks for the question. We use the BN layer in the network and it can be eliminated in the inference. At inference time, the batch statistics (mean and variance) are no longer needed. Instead, we can fold the normalization, scaling, and shifting operations into the weights and biases of the previous layer. Then $W′=σ/γ⋅W$ and $b′=γ(b−σ/μ)+β$. In our paper, we propose the re-parameterization method to fold the $σ/γ$ into the previous layer activation (see Eq.16). Thus the BN can be removed still in the ReverB.\n\n**Question 3**: Why does ReverB exhibit performance advantages over traditional SNNs, and can the authors provide an explanation for this?\n\n**A3**: Thanks for the question. This is because ReverB quantizes the weight while traditional SNNs quantize activation. Highlight greater accuracy degradation from quantizing activations compared to weights. This is due to several key reasons:\n\n**First**, activations often have a much wider and more varied dynamic range compared to weights. Due to the large dynamic range and possible non-uniform distribution of activations, quantizing them requires compressing a broader set of values into a smaller bit-width, which can result in greater precision loss.\n\n**Second**, Activations change frequently during the forward pass, as they are directly influenced by the input data and the weights. The values can vary significantly between different layers, and this variability can be much greater in deeper layers. Such frequent and complex changes make activations more sensitive to quantization errors. On the other hand, weights are stable and unrelated to input, making them less prone to large errors when quantized.\n\n**Third**, Here, we will also use the entropy concept to show that ReverB-SNN has a higher information capacity than vanilla SNN. thus exhibiting performance advantages over traditional SNNs.\n\nWe first perform a theoretical analysis using the concept of entropy. The representational capability $ \\mathcal{C}(\\mathbf{X}) $ of a set $ \\mathbf{X} $ is determined by the maximum entropy of $ \\mathbf{X} $, expressed as:\n$\\mathcal{C}(\\mathbf{X}) = \\max \\mathcal{H}(\\mathbf{X}) = - \\sum_{x \\in \\mathbf{X}} p_{\\mathbf{X}}(x) \\log p_{\\mathbf{X}}(x),$ where $ p_{\\mathbf{X}}(x) $ is the probability of a sample $ x $ from $ \\mathbf{X} $. Then it is clear for the following proposition:\n\nFor a set $ \\mathbf{X} $, when the probability distribution of $ \\mathbf{X} $ is uniform, i.e., $ p_{\\mathbf{X}}(x) = \\frac{1}{N} $, where $ N $ is the total number of samples in $ \\mathbf{X} $, the entropy $ \\mathcal{H}(\\mathbf{X}) $ reaches its maximum value of $ \\log(N) $. Hence, we conclude that $ \\mathcal{C}(\\mathbf{X}) = \\log(N) $.\n\nUsing the proposition, we can evaluate the representational capacity of vanilla SNN and our model.\n\nLet's consider two connected neuron layers. Since the connectivity of the neurons is the same no matter for vanilla SNN or our model, let us focus on an arbitrary two connected neurons from different layers. \n\nFor the vanilla SNN, the values output from one neuron to another are {0, 1} x v, where v is the fixed weight between the two neurons. Thus one neuron from vanilla SNN could transform two samples into another one, and $ \\mathcal{C}(\\mathbf{X}) = \\log(2) = 1$. \n\nFor our model, the values output from one neuron to another are u x {-1, 1}, where u is the real-valued spike and could be changed. u requires 32-bits. Thus the number of possible samples from u is 2^{32} and the value samples from one neuron to another is 2^{32+1}. $ \\mathcal{C}(\\mathbf{X}) = \\log(2^{32+1}) = 33$.\n\nThis highlights the limited representational capacity of vanilla SNN compared to our model.", "claims_and_evidence": "Yes, the claims made in the submission are supported by clear and convincing evidence. The method is evalued across various network architectures and datasets.", "methods_and_evaluation_criteria": "The method proposed in this paper is effective. The new network, positioned between BNNs and SNNs, is expected to fully leverage the advantages of both, and potentially start a new field in lightweight networks. The extensive experimental validation across diverse architectures and datasets convincingly demonstrates the superiority of this method over current state-of-the-art approaches.", "theoretical_claims": "Not applicable. This paper does not involve complex theoretical proofs.", "experimental_designs_or_analyses": "The experimental design is generally reasonable. The method is evalued across various network architectures and datasets. The authors also conducted a series of ablation experiments to evaluate the effectiveness of the proposed method.", "relation_to_broader_scientific_literature": "This paper introduces a novel binary design in SNN termed ReverB, which uses real-value activation and binary weights, merging the characteristics of both BNN and SNN. It potentially starts a new field in lightweight networks.", "essential_references_not_discussed": "I think the paper has cited enough relevant literatures.", "other_strengths_and_weaknesses": "The paper is well-written, the idea is interesting, and the results are impressive.", "other_comments_or_suggestions": "None", "questions_for_authors": "1.Does the ReverB network has a reset mechanism, and how does it function?\n2.Does the ReverB network use the Batch Normalization (BN) layer, and can ReverB eliminate it during inference similar to SNN?\n3.Why does ReverB exhibit performance advantages over traditional SNNs, and can the authors provide an explanation for this?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper proposes a novel spiking neural network (SNN) paradigm that inverts the conventional quantization approach by employing ternary weights and real-valued spikes, significantly enhancing network expressiveness while preserving energy efficiency. The method demonstrates competitive classification accuracy across various architectures (ResNet, VGG) and datasets (CIFAR, ImageNet), with ablation studies validating its design. While reviewers raised concerns regarding hardware compatibility analysis, hyperparameter optimization transparency, and missing gradient derivations, the core contribution—enhancing SNN expressivity through weight-spike role reversal—remains valid. The authors have addressed major concerns during rebuttal, and remaining issues (e.g., energy evaluation methodology, literature positioning) can be resolved through revisions. Therefore, an acceptance is recommended.
Decision: Accept (poster)