BSO: Binary Spiking Online Optimization Algorithm
Authors: Yu Liang
Yu Yang
Wenjie Wei
Ammar Belatreche
Shuai Wang
Malu Zhang
Yang Yang
Venue: ICML 2025 poster (2025)
Keywords: Spiking Neural Network
Online Learning
Spiking Binarization
Primary area: applications->neuroscience_cognitive_science

Abstract: Binary Spiking Neural Networks (BSNNs) offer promising efficiency advantages for resource-constrained computing. However, their training algorithms often require substantial memory overhead due to latent weights storage and temporal processing requirements. To address this issue, we propose Binary Spiking Online (BSO) optimization algorithm, a novel online training algorithm that significantly reduces training memory. BSO directly updates weights through flip signals under the online training framework. These signals are triggered when the product of gradient momentum and weights exceeds a threshold, eliminating the need for latent weights during training. To enhance performance, we propose T-BSO, a temporal-aware variant that leverages the inherent temporal dynamics of BSNNs by capturing gradient information across time steps for adaptive threshold adjustment. Theoretical analysis establishes convergence guarantees for both BSO and T-BSO, with formal regret bounds characterizing their convergence rates. Extensive experiments demonstrate that both BSO and T-BSO achieve superior optimization performance compared to existing training methods for BSNNs. The codes are available at \url{https://github.com/hamingsi/BSO}.

BSO: Binary Spiking Online Optimization Algorithm
Yu Liang'
Yu Yang
Wenjie Wei
Ammar Belatreche?
Shuai Wang' Malu Zhang'
Yang Yang
Abstract
promising solution for resource-constrained edge com- puting scenarios (Srinivasan & Roy, 2019; Lu & Sengupta, 2020; Kheradpisheh et al., 2022). Despite these notable ad- vantages, BSNNs predominantly utilize training algorithms inherited from full-precision SNNs.
Binary Spiking Neural Networks (BSNNs) of- fer promising efficiency advantages for resource- constrained computing. However, their training algorithms often require substantial memory over- head due to latent weights storage and temporal processing requirements. To address this issue, we propose Binary Spiking Online (BSO) opti- mization algorithm, novel online training algo- rithm that significantly reduces training memory. BSO directly updates weights through flip sig- nals under the online training framework. These signals are triggered when the product of gra- dient momentum and weights exceeds thresh- old, eliminating the need for latent weights dur- ing training. To enhance performance, we pro- pose T-BSO, a temporal-aware variant that lever- ages the inherent temporal dynamics of BSNNs by capturing gradient information across time steps for adaptive threshold adjustment. The- oretical analysis establishes convergence guar- antees for both BSO and T-BSO, with formal regret bounds characterizing their convergence rates. Extensive experiments demonstrate that both BSO and T-BSO achieve superior optimiza- tion performance compared to existing training methods for BSNNs. The codes are available at /ttp::///itttnn com hwwingsiinnnnnniimiii.iio /BSC.
Current BSNN training methods fall into two main cate- gories: ANN-SNN conversion and direct training. ANN- SNN conversion obtains BSNN weights from pre-trained Artificial Neural Networks (ANNs) (Rueckauer et al., 2017; Roy et al., 2019a; Wang et al., 2020; Yoo & Jeong, 2023). However, these approaches typically require long simulation time steps to match ANN performance, inevitably increasing energy consumption. Moreover, cannot handle sequen- tial data. To address these limitations, researchers have developed direct training algorithms for BSNNs, with surro- gate gradient- based backpropagation through time (BPTT) emerging as a predominant training paradigm (Deng et al., 2021; Pei et al., 2023; Wei et al., 2025). Unfortunately, this methodology requires maintaining both computational graphs and latent weights for BSNNs, resulting in signifi- cant memory and computation overhead. Consequently, this resource-intensee training process inherently contradicts the efficiency advantages of BSNNs. This motivates to raise an intriguing question: Can we leverage the substantial efficiency advantages of BSNNs not only during the forward pass but also throughout the learning process?
Recently, online training algorithms for SNNs have gained prominence for their temporal independence, which sub- stantially reduces training costs in memory (Xiao al., 2022; Meng et al., 2023; Jiang et al.). However, existing SNN online training algorithms are predominantly designed for general situations. Given the efficiency advantages of BSNNs in resource-constraing environments, the integra- tion of online training presents a promising direction in efficient neuromorphic computing. However, this integra- tion faces a fundamental challenge: the storage overhead of latent weights in existing BSNN training methods con- flicts with online training's memory efficiency objectives. This contradiction underscores the urgent need to develop specialized online training techniques tailored for BSNNs.
1. Introduction
Spiking Neural Networks (SNNs), as the third generation of neural network paradigms (Maass, 1997; Roy et al., 2019b), have garnered significant attention in machine learning at- tributed to their bio-inspired mechanisms and energy effi- ciency advantages. Building upon this foundation, Binary SNNs (BSNNs) further enhance computational efficiency by incorporating binary weights representations, offering
University of Electronic Science and Technology of China NNottuubrria University. Correspondence to: Malu Zhang <maluzhang @ uestc.edu.cn...
To address these challenges, we propose a Binary Spik- ing Online (BSO) optimization algorithm that leverages BSNN's efficiency advantages in both forward and back-
Proceedings of the 4nnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

BSO: Binary Spiking Online Optimization Algorithm
ward propagation. BSO offers two key advantages: first, its memory requirements are orthogonal to time steps, sig- nificantly reducing memory and computational overhead; second, it updates weights through sign flips, eliminating the need for latent weights. Furthermore, to enhance per- formance by leveraging the inherent temporal dynamics of SNNs, we introduce a temporal-aware variant called T-BSO, which effectively integrates gradient information across each time step.
thresholds, proposing a weight-threshole balancing conver- sion method to reduce errors during conversion. However, these converted SNNs suffer from performance degradation and long latency. To address these limitations, researchers have developed direct training methods for BSNNs. (Qiao et al., 2021) employ a surrogate gradient method to train a hrrinlye-friendly BSNN for processing temporal neuromor- phic data. (Kheradpisheh et al., 2022) introduce BS4NN, a temporal-encoded BSNN where each neuron fires at most one spike, providing energy-efficient event-driven learn- ing approach. To further reduce the quantization error and improve performance, some studies have introduced differ- ent strategies to enhance performance, such as alternating direction methods (Deng et al., 2021), accuracy loss estima- tor (Pei et al., 2023), improved activation function (Hu et al., 2024), weight-spike regulation (Wei et al., 2024; Wang et al., 2025), and adaptive gradient modulation (Liang al., 2025).
Contributions. In summary, our key contributions are sum- marized as follows:
We introduce BSO, the first online training optimiza- tion algorithm tailored for BSNNs. BSO significantly reduces memory overhead through two advantages: (a) ensuring memory requirements orthogonal to time steps and (b) eliminating latent weights storage via gradient-dominated flipping.
Recently, researchers have also designed quantization strate- gies for spiking transformers, achieving competitive results on vision tasks (Qiu et al., 2025; Cao et al., 2025).
We propose T-BSO, an enhanced version of BSO that significantly improves performance by effectively exploiting SNNs' temporal dynamics, capturing the second-order gradient moment across time steps for adaptive threshold optimization.
As described above, existing learning algorithms for BSNNs are adapted from full-precision SNNs. While these methods leverage BSNN's energy efficiency in forward propagation, they fail to exploit its binary weight characteristics during the training process. This motivates us to design a dedicated training algorithm for BSNN that leverages the weight sign flipping property for parameter updates and climinates latent weight reliance, enabling energy efficiency in both forward and backward propagation.
We rigorously prove the convergence of both BSO and T-BSO algorithms by deriving their theoretical regret bounds. Our comprehensive analysis demonstrates that the parameter updates in BSO and T-BSO maintain substantial directional consistency with convergence.
2.2. Online Training Method in SNNs
Our BSO and T-BSO demonstrate superior perfor- mance on both large-scale static and neuromorphic datasets across CIFAR-10, CIFAR-100, ImageNet, and CIFARIO-DVS, Experimental results verify time- independent training memory requirements, substan- tially reducing training overhead.
In recent years, significant progress has been made in online training algorithms for SNNs to achieve memory and online training (Zenke & Ganguli, 2018; Bellec et al., 2018; Bohnstingl et al., 2022). Kaiser et al. (2020) proposes local training methods that disregard temporal dependen- cies while Yin et al. (2023) adapts the approach from (Kag & Saligrama, 2021) using gated neuron models. Notably, OTTT (Xiao et al., 2022) extends the applicability of on- line training methods to address large-scale computational tasks. SLTT (Meng et al., 2023) establishes that tempo- ral domain gradients contribute minimally to SNN train- ing and proposes their elimination for improved efficiency. NDOT (Jiang et al.) introduces novel gradient estimation techniques based on neuronal dynamic algorithm. These approaches collectively offer diverse solutions for reducing memory overhead during SNN training. However, exist- ing online training methods predominantly focus on full- precision SNNs, leaving notable research gap in effectively integrating online training with BSNNs. This limitation sig- nificantly constrains the training and deployment of BSNNs in resource-constrained scenarios, particularly in practical applications demanding a low memory footprint.
2. Related Work
2.1. Learning algorithm for BSNNs
Training algorithms for BSNNs fall into two main cate- gories: ANN-SNN conversion methods and direct training methods. The ANN-SNN conversion method trains an ANN first, then directly transfers the weights to BSNN. (Roy et al., 2019a) first analyze the combination of different bi- nary neurons and binarized weight methods to train an ANN, and then obtain deep SNN with binary stochastic activa- tions through a conversion approach. Similarly, (Lu & Sen- gupta, 2020) utilize standard training techniques to train a binary convolutional model, then generate a binary SNN via a conversion process. Furthermore, (Wang et al., 2020) ex- amine the relationship between weights and spiking neuron
2

---

BSO: Binary Spiking Online Optimization Algorithm
3. Preliminaries
Inference Process Binary SNNs
Propagation
3.1. Binary Spiking Neural Networks
Forward
LIF
Spiking neurons form the basis of neural information trans- mission through spike trains. In BSNNs, these neurons replicate the behavior of biologicial neuron which inte- grates input spikes into its membrane potential and fires spike only when exceeds threshold. We consider the Leaky Integrate-anc and-Fire (LIF) model, which describes the dynamics of the membrane potentials as follows:
Spike S
Binary Wp
Presynaptic Input
Update Process of Binary Weight wa
Badeward Propagation
-01
+22
Latent Weights
Momentum
Newly Weights
du(t) -u((() Urest) + R- I(1), u(l)= Veh, (1) dt
x
Convolution Operation
Forward Process
Backward Process
Figure I. Forward and backward propagation of BSNNs under BPTT method.
where (tt) is the input current, Vvs is the threshold and 7 and R are resistance and time constant respectively. A spike is generated when u(t) reaches VeA at time e,, and u(t) is reset to the resting potential Uyests which is often zero. The spike train is defined using the Dirac delta function: s(t) = E, t").
3.2. Online Training Framework
BPTT method unfolds the iterative update equation in Eq. and backpropagates along the computational graph. The gradients with T time steps are calculated by:
Spiking neural networks comprise multiple layers of in- terconnected neurons with associated weights. The input current at layer I and time t follows [t] = = =W'' where W represents connection weight and [1) denotes spike outputs from the previous layer. Typically, the discrete method is employed to discretize the dynamic equations of LIF models, resulting in the following iterative form:
OC owe =
T
Ou'lt) aW:
W' [t],
Σ 1/1
EI
ket i=
Ou'li + 1] Os
Ou' l + 11
Oul
Os
Oul [
Du
owi
(5)
where C is the loss, W the connection weight from and ('(4) e ac 0x'l
ut [t] A(u' 1] Vn - 1J) + W w's'- s' [t] (u
layer 1
is the gradient for as du'((]
(3)
0s will be replaced u [t]. The non-differen non-differentiable terms au'(t by surrogate functions, i.c. derivatives of rectangular or sigmoid functions (Wu et al., 2018).
where H(-) is the Heaviside step function, s'{(] and u' [t] are the spike train and membrane potential at discrete time-
step / for layer and À is leaky term (typically taken
For online training framework, we decouple the full gra- dients into temporal components and spatial components to enable the online gradient calculation. All temporal de- pendencies are primarily manifested in the spiking neuron dynamics, i.e. (u'(i+1] de'()) define the instantaneous gradient at time / as:
1 -
)). The reset operation is implemented by subtracting the threshold Vnn-
To achieve significant model compression, previous BSNNs methods adopt the sign function during the forward pro- cess to convert W in (2) into binary representations. Therefore, the binarization on w E W can be formulated:
[B*( au'l and @u'[i+1] in E5, We "bu'l]
Lw = c'
(6)
= { -1, w < O,
w₂ = sign(w)
= +1, otherwise,
(4)
where a'(t] denotes the presynaptic activities that can be tracked recursively during forward propagation:
where wh is binary weight. Note that the real-valued weights w are maintained as latent weights to accumu- late couudo-gradints.. In the backward pass, the straight- through estimator (STE) is employed to approximate the gradient of the sign function (Bengio et al., 2013), i.e., Bsign(-) = 1 1 However, maintaining latent weights BW
a'(t] = [t]ā - 1] + ss(]
(7)
When p'[t] is the constant X, the OTTT (Xiao et al., 2022) ='st]_Vase'(1) leads to the
is obtained, while p'[t] derivation of the NDOT (Jiang et al.). The total loss L is defined as the sum of instantaneous losses over time steps:
incurs additional memory overhead and complicates the optimization process of BSNNs, which will be discussed in Section 4.1.
T L Σ L4 = - C C6 14,1),
(8)
3

---

BSO: Binary Spiking Online Optimization Algorithm
(a) The Backpropagation of BSO method
(b)
The Backpropagation of T-BSO method
W
M
W
W
M
W
G
G
Y1
Y
0.1
42
-111
G'[T)
G [T]
YT
M First order momentum
G [t]: Gradient matrix at time
Temporal-aware flipping threshold
Yt y
[t]: Second order momentum
y: Weight sign flipping threshold
Weight update strategies of BSO and T-BSO during backpropagation. (a) BSO employs the same flipping threshold y each time step, updating synapse weights through the dot product of momentum M and Binary W in BSNNs. (b) T-BSO incorporates second-order momentum e'[t] into y, thereby achieving temporal dynamic allocation with Temporal aware aware flipping threshold "r..
Figure 2.
(()) can be the cross-entropy loss, and y is the label. Under this framework, parameters can be updated either immedi- ately at each time step (real-time update) or after accumu- lating gradients over all time steps (accumulated update):
as indicators of sign flipping, and their precise values are not that important for the update mechanism.
This observation reveals fundamental critical insight: net- work behavior modifications in BSNNs are fundamentally governed by the frequency and timing of weight sign flips rather than by the continuous optimization of latent weight magnitudes. Consequently, we can explore innovative weight update strategies that directly generate flip signals. This approach bypasses latent weight representations en- tirely and thereby significantly reduces BSNN training mem- ory overhead. One intuitive approach involves comparing gradient magnitudes against fixed thresholds to generate flip signals. Therefore, this motivates us to explore more stable and efficient optimization strategies that better align with the inherent spatiotemporal characteristics BSNNs.
W t W V L[t] (real-time),
(9)
W W'
7 V Ltt (accumulated),
(10)
where is the learning rate. This framework enables forward-in-time learning with only constant memory costs, avoiding the significant memory consumption.
4. Method
In this section, we first analyze the behavior of latent weights in BSNNs and introduce the BSO optimization algorithm. Secondly, drawing inspiration from the temporal dynamics of neurons, we propose the Temporal-aware BSO to cap- ture temporal gradient information. Finally, we provide analysis of convergence and computational complexity.
4.2. Binary Spiking Online Optimization
As discussed in Section 4.1, we aim to completely eliminate latent weights to further significantly reduce the training memory overhead of BSNNs. To this end, we propose novel momentum- gradient accumulation strategy combined with threshold-controlledd weight updates: BSO algorithm. It leverages momentum to accumulate instanta- neous gradients and generate stable weight sign flip signals. At each time step t, BSO updates the momentum M for layer based on the computed instantaneous gradient:
4.1. Observation from Latent Weights
To ensure compatibility with existing backpropagation frameworks (e.g.. BPTT, OTTT), BSNNs employ latent weights and momentum (Sutskever al., 2013) guide binary weight sign updates during training. However, due to the binary paradigm of BSNNs weights, the actual up- date process only determines which weight signs require flipping, independent of the specific magnitudes latent weights. As shown in Figure the latent weights serve only
M B M.MM- + + 3) G.(t),
(11)
where G' = w L(t] and the hyper- parameter 3 control the exponential decay rates of the momentum. When the

---

BSO: Binary Spiking Online Optimization Algorithm
value of is relatively large, greater weight is assigned to historical gradients, making the momentum less sensitive to the gradients in the current time step. The weight update mechanism is guided by the following equation:
first and second order moment of the gradient as follows:
M + 31 M + (1 - B1) G(t),
(13)
1' + 32 w (1 -1-1 32)
meannG?(t)),
where the mean function is operated at each layer / and hyper-parameters B1, 82 are the decay rates for momentum. We define the temporal-aware threshold as T [ where e is a small constant to prevent zero threshold, thereby enhancing training stability. It accounts for the inherent temporal characteristies of BSNNs by considering gradients across different time steps. This variant avoids the linear growth of training memory cost with time steps by averaging gradients across different time steps. The weight update rule will be transferd as:
W + -iign(" O M' - 7) @ W
(12)
As shown in Fig.2, flip signal is triggered only when the element-wise product of W' and M' exceeds the thresh- old y. ensuring alignment between the weight and gradient momentum directions. This selective flipping mechanism enhances optimization stability by flipping weights only when their direction aligns with the gradient momentum. Conversely, when their signs differ, the weight sign is pre- served to prevent updates along gradient ascent directions, thereby avoiding potential optimization divergence.
W - - sign o M - 7/
[+ 3
(14)
Compared to BPTT, BSO eliminates the need for storing extensive computational histories and completely removes dependency on latent variables. This dual optimization approach enables BSO to achieve both computational effi- ciency and significantly reduced resource consumption dur- ing training. However, BSO's 's uniform gradient treatment across time steps overlooks valuable temporal information, leading to the loss of BSNN's inherent temporal dynamics. Therefore, this motivates us to further explore advanced extensions of BSO for temporal dynamics.
Second-order moments e'it] are crucial for capturing gra- dient magnitude variations across temporal dynamics in BSNNs. While first-order moments provide directional guid- ance, second-order moments adapt to the scale and variance of gradients at each time step. This temporal adaptation en- ables more precise threshold estimation through V + which automatically decreases in regions with consistently small gradients to facilitate weight flipping, while increas- ing high-gradient areas to prevent oscillatory updates. This mechanism effectively responds to the heterogeneous gradient distributions that naturally occur across different time steps in spiking neural networks.
4.3. Temporal-aware BSO
As described above, BSNNs exhibit temporal dynamics where neuronal states create dependencies between past and current inputs. Each time step generates unique components from presynaptic activity ä a'[t]. causing temporal variation in gradient distributions. Our analysis in Figure 3 confirms this variability is driven by spiking activity and historical states, highlighting the importance of considering individual time step contributions for flip signal generation. While M captures gradient information during training, extending it to temporal-awer representation M' would better account for SNN dynamics. However, this approach incurs memory costs that scale linearly with time steps. Inspired by adaptive algorithms, we propose a temporal variant to address this challenge: Temporal-aware BSO. The algorithm update the
Compared to BSO, T-BSO algorithm significantly enhances performance by incorporating temporal gradient dynamics to intelligently control weight sign flips. A key advantage of T-BSO is its highly efficient memory ietraationndsspiii tracking temporal information, it effectively maintains com- putational efficiency by averaging second-order moments across time steps. The comprehensive details of our ap- proach are provided in Algorithm 1.
4.4. Convergence and Complexity Analysis
Convergence analysis We analyze the convergence of BSO and T-BSO using the online learning framework pro- posed in (Zinkevich, 2003). Consider an online optimization scenario where at each round E 1, ..., T, an algorithm predicts the parameter ww and incurs loss based on a con- vex cost function fw) The sequence ff TT= is arbitrary and unknown in advance.
10
0.0
Theorem 4.1. Assume that the function fo has bounded gradients, [[ (w)ll2 S G. [[(((( S Goo for all w E Edd. Let y and decay by Vb. BSO achieves the following guarantee for all l >
17
- 0.5
3
5
2
-1.0
(a) CIFAR-10
(b) C1F
(c) ImageNet
T
T
Figure 3. The cosine similarity between the gradients across time step. Both horizontal and vertical axes represent time sleps.
Ry <
2
Tk
+2
Σ
Ma = O(VT).
(15)
R
km
5

---

BSO: Binary Spiking Online Optimization Algorithm
O(n*L) storage cost for latent weights; although OTTT is time-independent, still needs latent weights to the same extent as BPTT. In contrast, BSO and T-BSO significantly reduce weight storage requirements by eliminating the need for full-precision latent weights. Building on this approach, BSO achieves time-independent memory costs by leverag- ing an online training framework. Additionally, T-BSO incorporates neglect variables into the optimizer to capture temporal dynamics (T < n²))
Table I. Memory complexity of BSNNs training methods.
METHODS
WEIGHT STORAGE
COMPUTATION STATE
BPTT
O(n²L)
O + nLT)
OTTT
O(n²L)
O(n nL)
BSO
O(n nL)
L/32)
T-BSO
O(n LT nL.
Let 61,32 and y decay by Vk, the T-BSO achieves the following guarantee for all k 2 L.
Algorithm BSO and T-BSO for optimization.
I: Input: T Number of time steps; Small constant to prevent zero threshold; B1, 52 Decay rates for mo-
T
mentum; Threshold; L Loss function; W Binary network weights at layer 1; x: Input data; y Ground
RT 2
7h
k=1
(16)
truth labels
Output: Optimized binary weights W
Our Theorem 4.1 demonstrates that when the gradients of BSNNs are bounded, the summation terms in BSO and T-BSO are respectively bounded by vT and T3/4, The gradual decay of $1, $2, and is essential for our theo- retical analysis and is consistent with previous empirical findings. For example, Sutskever et al. (2013) suggests that decreasing the momentum coefficient towards the end of training can improve convergence.
3: M < 0 Initialize 1 1st moment vector
4: of 0 Initialize temporal 2nd moment vector
5: while not converged do
6: for t + to 7 do
7: for I + to N do
u'[1] < )(u'(t- 1--n- (t-))) W
s'( k H( V(n)
10:
a'[ + Aá
+ +
Corollary 4.2. Assume that the function Sú. has bounded
11:
end for
gradients, [Ve((l)) S G,IlVf&(W)) S Goe for all
12:
for 1 +- N downto do
Rd. BSO achieves the following gurantee, for all
13:
Git) < W L[t]
14:
M < 31 M + (1 3-) G[4
T 121 1.
R(T) = 0
15:
i-BSSO then
T VT
(17)
16:
I'[t] + 82 vf [t] + (1-32)- mcan (G²
17:
end if
while T-BSO achieves the following gurantee, for all T 2
1.
18:
if T-BO then
19:
W + sign W/ O -M (
R(T) T
(1T41/4)
(18)
20:
else
21:
W - Win GM (M -7) @ w
22:
end ir
This result can be obtained by using Theorem 4.1. Thus, I(T) O. More details proided in Ap- lim 7
23:
end for
24: end for
pendix A. The findings demonstrate that BSO and T-BSO algorithms will achieve convergence throughout the train- ing phase, provided that gradients remain bounded. The convergence analysis is crucial as it guarantees the stability and effectiveness of the optimization algorithms, ensuring that they will eventually reach an optimal or near-optimal solution as the training progresses.
25: end while
5. Experiments
In this section, we first present experimental details, includ- ing the utilized datasets, architectures, and settings. Second, we compare our methods with existing online training and BSNN approaches to evaluate their effectiveness. Thirdly, we conduct ablation studies to assess the efficiency improve- ments of BSO and T-BSO during the training process, as well as the performance advantages of T-BSO over BSO. Finally, we analyze the hyperparameter and scalability study of our more advanced T-BSO.
Complexity Analysis We analyze the memory require- ments for weight storage and computation state during train- ing, summarized in Table 1. Let n represent the average number of neurons per layer. BPTT with SG maintains the entire computational graph to enable backpropagation of gradients through time. Consequently, BPTT requires time- dependent memory for gradient computation and incurs an
6

---

BSO: Binary Spiking Online Optimization Algorithm
Table Comparisons BSO and T-BSO with related work, focusing the online training and BSNN methods in the SNN domain.
OLINE
BINARY
TIME
MODEL
Acc.
DATASET
METHOD
ARCHITECTURE
TRAINING
WEIGHT
STEP
SIZE (MB)
(%)
FP-SNN OTTT (XIAO AI.,, 2022)
VGG-11
X
X
36.91 BASE
93.23 BASE
VGG-11
X
6
36.91 (LIX)
93.73 (40.5)
NDOT (JIANG AL.)
VGG-11
X
6
36.91 (1.8)
94.90 (+1.7)
CIFAR- -0
CBP (Yoo & JEONG, 2023)
VGG-16
X
32
15.10 (2.4x)
91.51
Q-SNN (WEI 2024)
VGG-11
6
1.20 (30.8x)
93.20
92.98 (-0.3)
BSO
VGG-11
4
1.20 (30.8))
93.45 (+0.2)
2
93.30 (+0.1)
6
94.70 (+1..)
T-BSO
VGG-11
4
1.20 (30.8))
94.86
2
94.32 (+1.1)
FP-SNN
VGG-11
AXX
6
37.10 BASE
71.15 BASE
OTTT (XIAO ET AL.,, 2022)
VGG-11
6
37.10 (LBX)
71,11 (-0.04)
NDOT (JIANG ET AL.)
VGG-11
6
37.10 (1.0x)
76.61 (+5.5)
CIFAR- 100
CBP (Yoo& & JEONG, 2023)
VGG-16
32
16.60 (2.2%)
66.53 (-4.6)
Q-SNN (WEI AL.,, 2024)
VGG-11
6
1.39 (26.7x)
73.48 (42.3)
6
72.57 (41.4)
BSO
VGG-11
4
1.39 (26.7x)
72.34
72.15 (+1.0)
6
74.27
T-BSO
VGG-11
1.39 (26.7x)
73.82 (+2.7)
2
73.40 (42.3)
FP-SNN
RESNET-18
X
AXX
4
87.19 BASE
63.18 BASE
IMAGENET
SLTT (MENG ET AL., 2023)
RESNET-34
6
85.15 (L.X)
66. (43.0)
OTTT (XIAO ET AL., 2022)
RESNET-34
6
87.19 (L.X)
64.16 (+1.0)
CBP (Yoo& & JEONG, 2023)
RESNET-18
4.22 (20.7x)
54.34 (-8.8)
6
58,86 (-4.3)
T-BSO
RESNET-18
4.22 (20.7x)
57.76 (-5.4)
FP-SNN
VGG-11
X
10
36.91 nASH
73.90 naSH
DVS CIFARIO IO
SLTT (MENG AL., 2023)
VGG-11
x
10
34.32 (L11)
82.20 (+8.3)
NDOT (JIANG AL..)
VGG-11
X
10
36.91 (1.X)
77.50 (43.6)
CBP (Yoo & JEONG, 2023)
16CoNvIFC
16
1.33 (27.8x)
74.70 (40.8)
Q-SNN (WEI ET AL., 2024)
VGG-11
10
1.20 (30.8))
79.10 (+5.2)
BSO
VGG-II
10
1.20 (30.8x)
80.60 (+6.7)
T-BSO
VGG-11
10
1.20 (30.8x)
81.00 (47.1)
5.1. Implementation Details
initial learning rate is set to 0.1 and decays to 0 throughout the training process. Moreover, we set the threshold Va and the decay factor in the LIF to and 0.5. For the hyperparameters of our BSO and T-BSO, we set y to 10-7, while 31 and Ba provide the hyperparameter analysis for them in Sec. 5.4. first pre-train the
We validate our BSO and T-BSO on image classification tasks, including both static image datasets like CIFAR- 10 (Krizhevsky et al., 2009), CIFARIOR 100 (Krizhevsky et al., 2009), ImageNet (Deng et al., 2009), as well as the neuro- morphic dataset DVS- DVS-CIFARIO (Li et al., 2017). Regarding the architecture, and ResNet-18 (Liu et al., 2018), which are commonly used in the SNN domain (Xiao et al., 2022; Jiang et al.). Consistent with prior work (Rastegari et al., 2016; Qin al., 2020), we maintain full-precision weights in the first and final layers. As for the training configuration, we apply SGD with cosine annealing learning rate schedule. The
10 and 1 10-5 We
When experimenting the ImageNet, model using single time steps for 100 epochs, then fine-tune with T-BSO using 4-6 time steps for 30 epochs to accelerate training. Additional details are available in Appendix B.
VGG (64C3-128C3-AP2-2555
5.2. Comparison of Performance
We evaluate our BSO and T-BSO methods on static and neu- romorphic datasets and compare with related work, mainly

---

BSO: Binary Spiking Online Optimization Algorithm
focusing on online training methods (Xiao et al., 2022; Meng al., 2023; Jiang et al.) and BSNN methods (Wei et al., 2024; Yoo & Jeong, 2023).
time-invariant memory consumption and avoiding BPTT's quadratic memory overhead. We validate this through train- ing memory measurements on CIFAR-10 under identical configurations (backbone, batch size, etc.). As shown in Fig- ure 4, BPTT algorithms commonly used in existing BSNNs increase memory usage with time steps, whereas BSO and T- BSO maintain constant memory. Specifically, when the time step grows from to 35, the memory cost of BPTT increases from 2.86GB to 58. 18GB (20.3x), while our BSO and T- BSO both maintain constant memory usage. 2.9GB and 3. IGB, respectively. The minor overhead in T-BSO arises from its camporal-award module. These results demonstrate that our BSO and T-BSO algorithm significantly reduce memory requirements during training.
As shown in Table 2. on the CIFAR-10 dataset, T-BSO achieves 94.70%, 94.86% and 94.32% accuracy with 6, 4, and 2 time steps, respectively. With a reduction in model size by 30.76x, our approach achieves performance compa- rable to existing online training methods and even surpasses OTTT by 0.97%. For CIFAR- 100, T-BSO also demonstrates superior performance, achieving the accuracy of 74.27%, 73.82% and 73,40% with 6, 4 and 2 time steps. Compared to BPTT-based BSNN methods like Q-SNN, T-BSO achieves performance improvements of 0.79%, while requiring less training overhead. On the ImageNet, with the same time steps. T-BSO achieves an accuracy of 57.76%, which is 3.42% higher than the BPTT-basee CBP method. Although there is still gap between T-BSO and methods like SLTT and OTTT on this dataset, it's worth noting that T-BSO achieves these competitive results with a model size of only 4.22MB (reduced by 20.18x, 20.66 On the neuromor- phic DVS-CIFARIE 10, T-BSO achieves 81. 31.00% accuracy with 10 time steps, outperforming both existing online training and BSNN methods. These results demonstrate that our proposed BSO and T-BSO methods maintain the efficiency advantages of BSNNs in both forward and backward propa- gation, while achieving superior performance.
76.0
BSO
75 20
75.0
T-BSO
74.85
Test A.e, (%)
74.27
74.0
7342
73.0
12.57
72.0
71.0
4
6
10
12
Time Step
Figure 5. Performance comparison between our BSO and T-BSO under different time steps. T-BSO consistently outperforms BSO across time steps and exhibits larger increase over time steps.
5.3. Ablation Study
This part presents ablation studies on BSO and T-BSO. First, we compare the memory overhead during training with the widely used BPTT learning method in BSNNs, demonstrating the efficiency of our BSO and T-BSO in backpropagation. Second, we validate the effectiveness of the lewporal-aware mechanism by analyzing T-BSO's performance improvement over BSO.
Comparative analysis of our BSO and T-BSO algorithms. In T-BSO, we introduce the time-aware threshold to con- sider the time information when updating the weights. We evaluate the accuracy of BSO and T-BSO at different time steps using experiments on the CIFAR- 100 dataset and the VGG-11 network. The experimental results are shown in Figure 5. can be seen that both BSO and T-BSO show improved performance with the increase of time steps. In ad- dition, due to the better utilization of temporal information by the temporal-aware threshold, T-BSO always outper- forms BSO in each time step, and the performance increases significantly with the time step.
60
BPTT
BSO
Memory Cost (GB)
50
T-BSO
3.5
40
Increpsia liscarty
GBB
T-BSO : 3.1GB
55 15:2W61
30
BSO 2.9GB
20
2.5
10
Consstnn
5.4. Hyperparameter and Scalability Study of T-BSO
0
5
10
15
20
25
30
35
45
As discussed above, we demonstrate the training efficiency of BSO and T-BSO, as well as the superior performance of T-BSO. In this part, we conduct a more in-depth analysis of our advanced T-BSO learning algorithm, including the hyperparameter study and the scalability to non-vision tasks.
Time Step
Figure 4. Memory cost comparison between the widely used BPTT algorithm in BSNN and our BSO/T-BSO across varying time steps.
Training cost comparison for three BSNN learning meth- ods (i.e., BPTT vs. BSO vs. T-BSO). BSO and T- BSO eliminate backpropagation through time, achieving
Hyperparameter study of T-BSO. Given that T-BSO introduces new byperparameters, we analyze its sensitiv- ity to parameter variations, including threshold y and the
8

---

BSO: Binary Spiking Online Optimization Algorithm
Table Effect of the threshold y in T-BSO performance.
Table 5. Performance comparison of T-BSO on the GSC dataset.
ONLINE TRAINING
BINARY
TIME STEP
MODEL
ACC.
Y (x100
0.005
0.02
0.05
0.08
METHOD
WEIGHT
SIZE (MB)
(%)
ACCURACY (%)
73.75
73.33
73.40
73.24
Y (x100=)
0.1
0.15
0.2
5.0
ATIF
X
X
4
44.74
94.31
ACCURACY (%)
73.38
73.34
73.41
72.83
STS-T
X
X
4
6.24
95.18
T-BSO
V
/
4
1.26
96.12
Table Effect of decay rates for momentum in T-BSO, i.e., B1 and 32, on performance.
6. Conclusion
Y
1 P1
1 32
ACCURACY
This paper introduces BSO, novel online optimization al- gorithm designed specifically for BSNNs. Unlike existing training methods, BSO eliminates the need for such auxil- iary storage by directly updating binary weights through a flip-based mechanism guided by gradient momentum. This design preserves the efficiency advantages of BSNNs in both forward and backward processes. To further improve performance, the paper proposes T-BSO, temporal-awar extension of BSO that incorporates second-order gradient moments to dynamically adjust flipping thresholds across time steps. This enhancement captures the intrinsic temporal dynamics of spiking activity and leads to more stable and ef- fective optimization. Theoretical analysis establishes regret bounds for both BSO and T-BSO, confirming convergence under standard assumptions. Experiments demonstrate that BSO and T-BSO achieve competitive accuracy while signifi- cantly reducing training overhead, representing substantial advance toward efficient and scalable BSNN training.
1.5 x 10
73.22
1.5 10
1.0 x 10-6
73.31
5.0 x 10-?
1.5 x 10-6
73.29
1.5 x 10
73.17
1.0 10-3
10
73.40
1.5 x 10-5
73.34
decay rates for momentum (3, and (2). Experiments are performed using VGG-11 architecture on CIFAR-100 with T=2 time steps. The analysis of y is presented in Table 3. The threshold parameter 7 maintains stable performance across range from 5.0 x 10-8 to 5.0 x 10-5, with ac- curacy varying only between 72.83% and 73.41% %. This stability demonstrates low sensitivity to threshold selection. The analysis of decay rates for momentum shown in Table 4. Despite the wide range of decay rates settings, per- formance variations remain minimal. Optimal performance achieved with smaller decay rates (1 $1 1.5 x 10-4
1- 82 = 1.5 x 10-5), yielding accuracy of 73.29%. These results demonstrate that while T-BSO's design intro- duces several hyperparameters, it exhibits low sensitivity to hyperparameters. Such robustness is beneficial, as it guar- antees consistent performance across variety of settings.
A Acknowledgements
This work is supported in part by the National Natural Sci- ence Foundation of China (No. U2333211, U20B2063 and 622201 06008), in part by the Project of Sichuan Engineering Technology Research Center for Civil Avia- tion Flight Technology and Flight Safety (No. GY2024- 27D). in part by the Open Research Fund of the State Key Laboratory of Brain-Machine Intelligence, Zhejiang University (Grant No.BM12400020), in part by the Shen- zhen Science and Technology Program (Shenzhen Key Laboratory, Grant No. ZDSYS202300220011022222 part by the Shenzhen Science and Technology Research Fund (Fundamental Research Key Project, Grant No. JCYJ202208 18 8103001002) and in part by the Program for Guangdong Introducing Innovative and Enterpreneurial Teams, Grant No. 2023ZT10X044.
Scalability Study of T-BSO. Existing online training methods have been primarily validated on image tasks, leav- ing their scalability other domains unexplored. To address this limitation, we evaluate T-BSO's effectiveness audio the Google Speech
tasks by conducting experiments Command (GSC) dataset (Warden, 2018). We evaluate T- BSO by employing the VGG-11 backbone with T' = 4, as shown in Table 5. Our approach is benchmarked against existing SNN methods, specifically ATIF (Castagnetti et al., 2023) and STS-T (Wang al.. 2023). Our exploration on the GSC dataset demonstrates T-BSO's superior perfor- mance. T-BSO achieves the highest accuracy of 96. 12% while being the only method supporting online training. Additionally, T-BSO operates with 1-bit quantization and maintains the model size of 1. 1.26MB. These findings show that our approach extends well to non-visual tasks, high- lighting its applicability beyond image-based uses.
in
Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences our work, none which we feel must be specifically highlighted here.

---

BSO: Binary Spiking Online Optimization Algorithm
References
Kaiser, J., Mostafa, H., and Neftci, E. Synaptic plasticity
dynamies for deep continuous local learning (decolle). Frontiers in Neuroscience, 14:424, 2020.
Bellec, G., Salaj. D., Subramoney, A., Legenstein, R., and Maass, W. Long short-term memory and learning-to- learn in networks of spiking neurons. Advances in neural information processing systems, 31, 2018.
Kheradpisheh, S. R., Mirsadeghi, M., and Masquelier, T. Bs4nn: Binarized spiking neural networks with temporal coding and learning. Neural Processing Letters, 54(2): 1255 1273, 2022.
Bengio, Y., Léonard, N., and Courville, A. Estimating or propagating gradients through stochastic neurons for con- ditional computation. arXiv preprint arXiv:1308.3432, 2013.
Krizhevsky, A., Hinton, G., et al. Learning multiple layers of features from tiny images. 2009.
Bohnstingl, T., Woćniak, S., Pantazi, A., and Eleftheriou, E. Online spatio-temporal learning in deep neural networks. IEEE Transactions on Neural Networks and Learning Systems, 34(11) ()8411))88899999 8908, 2022.
Li, H., Liu. H., Ji, X., Li, G., and Shi, L. Cifar10-dvs: an event-stream dataset for object classification. Frontiers in neuroscience, 1 11:309, 2017.
Cao, H., Zhou, Z., Wei, W., Belatreche, A., Liang, Y., Zhang, D., Zhang, M., Yang, Y., and Li, H. Binary event-driven spiking transformer. arXiv preprint arXiv:2501 105904, 2025.
Liang. Y., Wei, W., Belatreche, A., Cao. H., Zhou, Z., Wang. S., Zhang, M., and Yang. Y. Towards accurate binary spiking neural networks: Learning with adaptive gradient modulation mechanism. In Proceedings of the AAAI Con- ference on Artificial Intelligence, volume 39, pp. 1402- 1410, 2025.
Castagnetti, A., Pegatoquet, A., and Miramond, B. Train- able quantization for speedy spiking neural networks. Frontiers in Neuroscience, 17:1 154241, 2023.
Liu, Z., Wu, B., Luo, W., Yang. X., Liu, W., and Cheng, K.- T. Bi-real net: Enhancing the performance of 1-bit cnns with improved representational capability and advanced training algorithm. In Proceedings of the European con- ference on computer vision (ECCV), pp. 722-737, 2018.
Deng, J., Dong, W., Socher, R., Li, L.-J,, Li, K., and Fei-Fei, L. Imagenet: A scage-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255. Iece, 2009.
Deng, L., Wu, Y., Hu, Y., Liang, L., Li, G., Hu, X., Ding, Y., compression using
Lu, S. and Sengupta, A. Exploring the connection between binary and spiking neural networks. Frontiers in neuro- science, 14:535, 2020.
Li, P., and Xie, Y. Comprehensive admm optimization and activity regularization. IEEE transactions on neural networks and learning systems, 34 (6):2791-2805. 2021.
Maass, W. Networks of spiking neurons: the third genera- tion of neural network models. Neural networks, 10(9): 1659-1671, 1997.
DeVries, T. Improved regularization of convolutional neural networks with cutout, arXiv preprint 2017.
Meng. Q., Xiao, M., Yan, S., Wang, Y., Lin, Z., and Luo, Z.- Q. Towards memory-and time-efficient backpropagation for training spiking neural networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6166-6176, 2023.
Fang, W., Yu, Z., Chen, Y., Masquelier, T., Huang, T., and Tian, Y. Incorporating learnable membrane time constant to enhance learning of spiking neural networks. In Pro- ceedings of the IEEE/CVF international conference on computer vision, pp. 2661-2671, 2021.
Pei, Y., Xu, C., Wu, Z., Liu, Y., and Yang, Y. Albsnn: ultra- low latency adaptive local binary spiking neural network with accuracy loss estimator. Frontiers in Neuroscience, 17:1225871, 2023.
Hu, Y., Zheng, Q., and Pan, G. Bitsnns: Revisiting energy- efficient spiking neural networks. IEEE Transactions on Cognitive and Developmental Systems, 2024.
Jiang, H., De Masi, G., Xiong, H., and Gu, B. Ndot: Neu- ronal dynamics-basee baase online training for spiking neural networks. In Forty-first International Conference on Ma- chine Learning.
Qiao, G., Ning, N., Zuo, Y., Hu, S., Yu, Q., and Liu, Y. Direct training of hardware-friendly weight binarized spiking neural network with surrogate gradient learning towards spatio-temporal event-based dynamic data recog- nition. Neurocomputing, 457:203-22 -2000 2021.
Kag, A. and Saligrama, V. Training recurrent neural net- works via forward propagation through time. In Interna- tional Conference on Machine Learning, pp. 5189-5200 PMLR, 2021.
Qin, H., Gong, R., Liu, X., Bai, X., Song, J., and Sebe, N. Binary neural networks: A survey. Pattern Recognition, 105:10 2020.
10

---

BSO: Binary Spiking Online Optimization Algorithm
Qiu, X., Zhang, M., Zhang, J., Wei, W., Cao, H., Guo, J., Zhu, R.-J., Shan, Y., Yang, Y., Li, H. Quantized spike- driven transformer. arXiv preprint arXtv:2501.33492, 2025.
Wei, W., Zhang, M., Zhou, Z., Belatreche, A., Shan, Y., Liang, Y., Cao, H., Zhang, J., and Yang, Y. Qp-snn: Quantized and pruned spiking neural networks. arXiv preprint arXiv:2502.05005, 2025.
Rastegari, M., Ordonez, V., Redmon, J., and Farhadi, A. Xnor-net: Imagenet classification using binary convo- lutional neural networks. In European conference on computer vision, pp. 525-542. Springer, 2016.
Wu, Y., Deng, L., Li, G., Zhu, J., and Shi, L. Spatio- temporal backpropagation for training high-performance spiking neural networks. Frontiers in neuroscience, 12: 331, 2018.
Roy, D., Chakraborty, L., and Roy, K. Scaling deep spiking neural networks with binary stochastic activations. In 2019 IEEE International Conference on Cognitive Com- puting (ICCC), pp. 50-58. IEEE, 2019a.
Xiao, M., Meng, Q. Zhang, Z., He, D., and Lin, Z. On- line training through time for spiking neural networks. Advances in neural information processing systems, 35: 20717-20730, 2022.
Roy, K., Jaiswal, A., and Panda, Towards spike- based ma- chine intelligence with neuromorphic computing, Nature, 575(7784):607-6111 2019b.
Yin, B., Corradi, F., and Bohté, M. Accurate online training of dynamical spiking neural networks through forward propagation through time. Nature Machine Intel- ligence, 5(5):518 (155:518-527, 2023.
Rueckauer, B., Lungu, I.-A., Hu, Y., Pfeiffer, M., and Liu, S.-C. Conversion of continuous- deep networks to efficient event-driven networks for image classification. Frontiers in neuroscience, 11:62, 2017.
Yoo, D. and Jeong, D. S. Cbp-qsnn: Spiking neural networks quantized using constrained backpropagation. IEEE Journal on Emerging and Selected Topics in Cir- cuits and Systems, 2023.
Srinivasan, G. and Roy, Restocnet: Residual stochastic binary convolutional spiking neural network for memory- efficient neuromorphic computing. Frontiers in neuro- science, 13:189, 2019.
Zenke, and Ganguli, S. Superspike: Supervised learning in multilayer spiking neural networks. Neural computa- tion, 3016):151-1141, 2018.
Zinkevich, M. Online convex programming and generalized infinitesimal gradient ascent. In Proceedings of the 20th international conference on machine learning (icml-03), pp. 928-936, 2003.
Sutskever, I., Martens, J., Dahl, G., and Hinton, G. On the importance of initialization and momentum deep learn- ing. In International conference on machine learning, pp- 1139-1147, PMLR, 2013.
Wang. S., Zhang, D., Belatreche, A., Xiao, Y., Qing, H., Wei, W., Zhang. M., and Yang, Y. Ternary spike-based neu- romorphic signal processing system. Neural Networks, 187:107333, 2025.
Wang, Y., Xu, Y., Yan, R., and Tang, H. Deep spiking neural networks with binary weights for object recogni- tion. IEEE Transactions Cognitive and Developmental Systems, 13(3):514-533 523,, 2020.
Wang, Y., Shi, K., Lu, C., Liu, Y., Zhang, M., and Qu, H. Spatial-temporal self-attention for asynchronous spiking neural networks. In ICCA,, pp. 3085-3093, 2023.
Warden, Speech commands: dataset for limited-vocabulary speech recognition. arXiv preprint arXiv:1804.0322 2018.
Wei, W. Liang, Y., Belatreche, A., Xiao, Y., Cao, H., Ren, Z. Wang, G., Zhang, M., and Yang, Y. Q-snns Quantized spiking neural networks. In Proceedings of the 32nd ACM International Conference on Multimedia, pp. 8441-8450, 2024.

---

BSO: Binary Spiking Online Optimization Algorithm
A. Convergence Proof
In this section, we prove Theorem 4.1.
Definition A.1. A function f R2 + convex for any 2, y E R, for all [0, 1].
(fa)) (1-A))(2) 2 s(rr + (1 A)y).
(19)
Also, notice that a convex function can be lower bounded by hyperplane at its tangent. Lemma A.2. If function f: Rd + R. is convex, then all 2, y R
f(v) 2 f(z) + Vf(x)) (y-))
(20)
The above lemma be used upper bound the regret and our proof for the main theorem is constructed by substituting the hyperplane with the BSO update rules.
Consider an online optimization scenario, where at each round à E 1.2..... T, an algorithm predicts the parameter wi and incurs a loss based on a convex cost function (w)). The sequence [ arbitrary and unknown in advance.
Definition A.3. The regret R(T) of an online algorithm over 'I rounds is defined as:
T R(T) Σ fw())
T Σ fw""),
min WEER
(21)
where A' is the feasible set of parameters.
Our objective is to analyze the regret of BSO and demonstrate that it achieves sublinear regret, implying that the average regret per time step diminishes as TT increases.
Assumption A.4 (Convexity). Each loss function fa (W) is convex with respect to W for all A: = 1,2,..., T. Assumption A.5 (Bounded Gradients). There exists a constant G such that for all k
V CWWIll S G.
Theorem A.6. Let fkr be function with bounded gradients such that [&x G and (w)) Goo for all E Rdd Furthermore, assume that any generated by BSO remains bounded. And the and B decay by Vk. Then BSO achieves a regret bound of R(T) = O(VT).
Proof. First, for each iteration k and flipped position i:
WR, - 7
(22)
the inner product decomposes
(Mx, Wv W Σ
M Mc, W7),
(23)
1=1
For each dimension i, since Wk. W E 1,
We,5 - W S Wa,l + WW $
(24)
For each iteration k:
(Ms, We We-WW) < |Ma W + +2/1; WK+1. 4 Wa.l
(25)
Consider the recursion:
B
M S Vk
G (1
M
(26)
Vk
By induction, this leads to:
M4 = O
(27)
VI
12

---

BSO: Binary Spiking Online Optimization Algorithm
Therefore:
T S
Ma 00
-
(28)
VT
1m
lm
Finally:
RT S 2
7k +
Σ
x O(VT).
(29)
k=1
Theorem A.7. Let J&. be function with bounded gradients such that [i(w)) S G and V&e(w)lo S Go for all w E RR. Furthermore, assume that any generated by BSO remains bounded. When Tkk and 31 82 decay by 1/V the algorithm achieves a regret bound of R(T) = O(T8/4).
Proof. First, we establish the moment bounds. For the first-order moment:
B1
31
Mr
Mk-1 1 + +
Gk.
(30)
Vk
Vk
Through recursion, obtain:
|M = -(11VA)
(31)
For the second-order moment:
B2
82
2k
2k- + (1-
Gh.
(32)
V
vk
Similarly:
|11/0 = O(1/Vi).
(33)
For the critical ratio, we derive:
/M&/ uk + c/
=
0(K-114).
(34)
At each iteration and flipped position i:
W WA. /Uk. 7h
~> 2 0.
(35)
The inner product decomposes as:
(Ma, Wv W*) Σ
Mc, - W7).
(36)
For each dimension i, since Wk,,, W E 11,1:
[K, - W.s 2
(37)
Therefore, at each iteration k:
Wx - W s |Ma We W + 2m/li: WA+1A * Wa.s-
(38)
From the moment bounds:
Σ
+
6/ S Σ =
(39)
Finally:
T
T
RT < 2
Th + 2
M V of + t O(T3/4).
(40)
1=1
k=1
13

---

BSO: Binary Spiking Online Optimization Algorithm
Table Training hyperparameters about BSO
Dataset
Epoch
Batch size
?
B
LR
CIFAR-I0
400
128
5e-7
1 1- 10-3
0.1
CIFAR-100
400
128
5e-7
1 1 1-10-3
0.1
DVS- DVS-CIFARIO
400
128
le-6
1 - 10-3
0.1
Table 7. Training hyperparameteers about T-BSO
Dataset
Epoch
Batch size
?
31
82
LR
CIFAR-10
400
128
5e-7
1-
1
1-10-
-003 3
1
10
0.1
CIFAR-100
400
128
5e-7
1
-110-
1-10-5
0.1
ImageNe((pre-train)
100
64
le-6
1-10-3
1-10-5
0.1
ImageNet(finetune)
30
64
le-11
1-10-
1- 1-10-5
0.1
DVS-CIFARIO
400
128
lc-6
1- 1-10-3
1 1- 10
0.1
B. Implementation Details
B.1. Datasets
We conduct experiments on CIFAR-10 (Krizhevsky et al., 2009), CIFAR-100 (Krizhevsky et al., 2009), ImageNet (Deng et al., 2009), CIFARIO-DVS (Li et al., 2017).
CIFAR-10 CIFAR-IO is a dataset consisting of color images across 10 object categories, with 50,000 training samples and 10,000 testing samples. Each image is 32 x 32 pixels with three color channels. We preprocess the data by normalizing the inputs using the global mean and standard deviation and apply data augmentation techniques including random cropping. horizontal flipping, and cutout Vries 2017). At each time step, the input to the first layer of the SNNs corresponds directly to the pixel values, which can be interpreted as real-valued input current.
CIFAR-100 CIFAR-100 is a dataset similar to CIFAR-10 but contains 100 object categories instead of 10. It includes 50,000 training samples and 10,000 testing samples, with the same preprocessing applied as in CIFAR-10 10. Both CIFAR-10 and CIFAR-100 are distributed under the MIT License.
ImageNet ImageNet-IK IK a large-scale dataset consisting of color images across 1,000 object categories, with 1,281,167 training samples and 50,000 validation images. We apply standard preprocessing techniques, where training images randomly resized and cropped to 224 x 224, followed by normalization after random horizontal flipping for data augmentation. For testing images, they are resized to 256 x 256, center-cropped to 224 x 224, and then normalized. At each time step, the inputs are converted into a real-valued input current.
DVS-CIFARI0 The DVS-CIFARIO dataset is neuromorphic version of the CIFAR-10 dataset, generated using Dynamic Vision Sensor (DVS). It contains 10,000 samples, one-sixth of the original CIFAR-10, and consists of spike trains with two channels corresponding ON- and OFF-event spikes. The pixel resolution expanded to 128 x 128. We split the dataset into 9,000 training samples and 1,000 testing samples. For data preprocessing, we reduce the time resolution by accumulating spike events (Fang et al., 2021) into 10 time steps and lower the spatial resolution to 48 x 48 through interpolation. We apply random cropping augmentation to the input data, similar to CIFAR-10, and normalize the inputs based on the global mean and standard deviation across all time steps.
B.2. Settings
For our proposed BSO and T-BSO algorithm, we apply the NDOT and OTTT for online training framework. The training hyper-parameters are detailed in Table and Table 7. Our BSO and T-BSO are tested with online update strategies, which updates the parameters in real-time at each time step to fully explore the potential of online training for BSNNs. For a fair comparison, we implement our Q-SNN (Wei et al., 2024) based on the VGG-11 11 architecture. For the GSC dataset, we employ a VGG-11 architecture with
10-7, B1 0.001, 32 = x 10-5 5 The GSC data are processed as follows: raw audio signals are converted Mel-spectrograms, which then resized to 32x32 pixels and normalized with mean of 0.5 and standard deviation of 0.5.
4. The training hyperparameters are set as:y
14


Reviews:
[{"review_id": "6LittKtipw", "reviewer": "Reviewer_Tbrf", "summary": "The paper proposes Binary Spiking Online Optimization (BSO) and its temporal variant T-BSO for training Binary Spiking Neural Networks (BSNNs) with reduced memory overhead. The work is well-motivated, technically sound, and demonstrates significant improvements in training efficiency and performance across static and neuromorphic datasets. While the core contributions are compelling, the paper would benefit from expanded discussions on related works and scalability to modern architectures like Transformers. With minor revisions, this work has the potential to advance resource-efficient neuromorphic computing.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for their thorough evaluation and valuable feedback. Our responses are provided below.\n\n**Response to Claim 2  \"statistical significance testing (e.g., standard deviations) is missing.\"**\n \nWe acknowledge that including standard deviations in Table 2 and Figure 4 would enhance the reproducibility of our results. We will revise the table and figure to include the standard deviations alongside the mean values to provide a clearer representation of the variability and ensure the results are more reproducible. \n\n**Response to Experimental Designs and W1: \"cite concurrent BSNN methods and compare with BSNN-online hybrid methods\", \"There could add more comparison with recent BSNN-online hybrid methods or other related advanced SNNs model.\"**\n\nThank you for pointing this out. We appreciate your suggestion to include comparisons with recent BSNN works such as [concurrent BSNN]. We would greatly appreciate it if the reviewer could kindly provide relevant references for these concurrent BSNN methods, which we would be happy to incorporate into our analysis. While our current baseline selection was based on a careful review of related methods at the time of submission, we recognize the importance of including more recent approaches to provide a comprehensive comparison. We consider extending BSO and T-BSO to more complex network structures like transformer-based and recurrent model to adapt to different application scenarios.\n\n**Response to W2 \"Discuss applicability to spiking Transformers.\"**\n\nRegarding the applicability to spiking Transformers, we believe our proposed method can be naturally extended to spiking Transformer-based models. In particular, our proposed T-BSO benefits from the adaptive threshold in the time dimension and can provide better convergence while preserving important **temporal information**. We extend our T-BSO to spikingformer[1] on CIFAR-10 with 4 timestep to prove the applicability. In particular, the binary spikingformer with T-BSO achieved an accuracy of 90.15%, indicating that the T-BSO algorithm can still ensure the **convergence** of binary online training in the transformer-like structure.\n\n[1]Zhou, Chenlin, et al. \"Spikingformer: Spike-driven residual learning for transformer-based spiking neural network.\" arXiv preprint arXiv:2304.11954 (2023).", "claims_and_evidence": "Yes, the claims are largely supported by evidence:\n\nClaim 1: BSO/T-BSO reduce memory overhead. Table 1 and Figure 5 convincingly show time-independent memory costs (e.g., 3GB vs. Q-SNN’s linear scaling).\n\nClaim 2: Superior performance. Table 2 validates T-BSO’s accuracy gains (e.g., 94.70% on CIFAR-10 vs. OTTT’s 93.73%), though statistical significance testing (e.g., standard deviations) is missing.\n\nTheoretical claims: Regret bounds (Theorem 4.1) are rigorously proven in Appendix A, assuming convexity and bounded gradients.", "methods_and_evaluation_criteria": "Yes, the methods and evaluations are appropriate:\n\nMethods: BSO’s flip-signal mechanism (Eq. 12) and T-BSO’s second-order momentum (Eq. 14) are novel and well-justified. The elimination of latent weights (Fig. 2) aligns with BSNN efficiency goals.\n\nEvaluation: Benchmarks (CIFAR, ImageNet, DVS-CIFAR10) are standard. Energy metrics (Table 2) and memory analysis (Fig. 5) effectively highlight efficiency gains.", "theoretical_claims": "Yes, theoretical analysis is correct under stated assumptions:", "experimental_designs_or_analyses": "Mostly sound:\n\nAblations: Table 3 and Figure 4 effectively validate T-BSO’s temporal adaptation.\n\nStatistical rigor: Missing standard deviations in Table 2 and Figure 4 reduce reproducibility.\n\nBaselines: Comparisons with OTTT, NDOT, and Q-SNN are thorough, but exclude recent BSNN works like [Cite concurrent BSNN methods].", "relation_to_broader_scientific_literature": "Strong.", "essential_references_not_discussed": "I suggest add more advanced SNNs models recently.", "other_strengths_and_weaknesses": "Strengths:\n\n1. The proposed method is novel.\n2. The technique is solid.\n3. The experiments are sufficient.\n\nWeakness:\n\n1. There could add more comparison with recent BSNN-online hybrid methods or other related advanced SNNs model.\n\n2. Discuss applicability to spiking Transformers.", "other_comments_or_suggestions": "None.", "questions_for_authors": "See weakness.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yYtCjfBYc2", "reviewer": "Reviewer_Q2xs", "summary": "The paper introduces Binary Spiking Online Optimization (BSO), a novel training algorithm for Binary Spiking Neural Networks (BSNNs) that significantly reduces memory overhead during training. The key innovations are two-fold:\n(1) making memory requirements independent of time steps, and\n(2) eliminating latent weight storage by directly updating binary weights through flip signals triggered when gradient momentum exceeds a threshold. \nThe authors also propose T-BSO, a temporal-aware variant that captures gradient information across time steps for adaptive threshold adjustment. Through theoretical analysis and experiments on datasets like CIFAR-10, CIFAR-100, ImageNet, and DVS-CIFAR10, the authors show that BSO and T-BSO achieve comparable performance compared to existing methods while substantially reducing training memory costs.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Response to W1. \"The T-BSO may introduce some additional computational complexity compared to the base BSO method.\"**\n\nThank you for your observation. It is true that T-BSO incorporates a lightweight second-order temporal gradient mechanism on top of the basic BSO method, potentially increasing computational overhead. Nonetheless, since this computation is averaged over the time dimension, it substantially enhances the network's **fitting capability** without significantly affecting the training time and memory usage. During inference, the temporal-aware threshold remains constant, thereby incurring only minimal computational overhead.\n\n**Response to W2. \"While promising, the results are primarily demonstrated on image classification tasks, and the method's performance on other domains remains to be explored.\"**\n\nIn this study, we primarily focus on image classification tasks to offer a clear and controlled evaluation of the proposed method. However, we acknowledge the importance of exploring the applicability of our approach to other domains, such as **speech recognition**. To this end, we train a ResNet19 model with T-BSO on the Google Speech Command (GSC) dataset [1] for speech recognition. The ResNet19 model with T-BSO was trained for 300 epochs on GSC's 35 categories, achieving an accuracy of 96.12%. This result demonstrates the potential of T-BSO in diverse application scenarios.\n\n**Response to W3. \"The proposed method introduces additional computational complexity through momentum-based weight flipping and adaptive thresholding.\"**\n\nThank you for pointing this out. In the proposed method, additional components such as momentum-based weight flipping and adaptive thresholds are introduced. Notably, the **latent weight** present in existing BSNN works is eliminated, and the subtraction operation is revised to flip signals, thereby enhancing the method’s lightweight and computationally efficient nature. \n\nThe momentum mechanism is based solely on simple element-wise operations, which impose minimal computational overhead, while the adaptive threshold does not necessitate extra backpropagation. Crucially, the increased complexity yields significant advantages in terms of performance enhancement, resulting in a favorable trade-off. Moreover, these additional computational operations are not executed during inference.\n\n**Response to W4. \"As a efficiency-focused work, why the authors did not compare efficiency metrics such as training or inference time, OP, SOP, or energy consumption.\"**\n\nThank you for your insightful comment. We fully agree that evaluating efficiency using metrics such as training/inference time, OPs, SOPs, or energy consumption would provide a more comprehensive understanding of the proposed method's advantages.\n\nThe table below provides a comparative analysis of the proposed method against SOP, OP, and energy with QSNN on the CIFAR-10 and CIFAR-100 datasets. We acknowledge that explicitly considering energy consumption and operation count is crucial for efficiency-oriented research, particularly in the context of spiking neural networks. Experimental results demonstrate that our BSO algorithm outperforms Q-SNN in both SOP and energy, owing to its lower spiking firing rate.\n\n| Dataset    | Algorithm | SOP                                      | OP                                      | Energy                                      |\n|------------|-------|------------------------------------------|-----------------------------------------|---------------------------------------------|\n| CIFAR-10   | BSO   | `0.93M(T=2)`, `1.51M(T=4)`, `1.69M(T=6)` | `1.78M` | `9.01uj(T=2)`, `9.54uj(T=4)`, `9.71uj(T=6)` |\n| CIFAR-10   | QSNN  | `1.70M(T=2)`, `2.80M(T=4)`, `3.88M(T=6)` | `1.78M` | `9.72uj(T=2)`, `10.71uj(T=4)`, `11.69uj(T=6)` |\n| CIFAR-100  | BSO   | `0.96M(T=2)`, `1.78M(T=4)`, `2.19M(T=6)` | `1.87M` | `9.48uj(T=2)`, `10.21uj(T=4)`, `10.59uj(T=6)` |\n| CIFAR-100  | QSNN  | `1.70M(T=2)`, `2.78M(T=4)`, `3.86M(T=6)` | `1.87M` | `10.13uj(T=2)`, `11.10uj(T=4)`, `12.08uj(T=6)` |\n\n[1]Warden, Pete. \"Speech commands: A dataset for limited-vocabulary speech recognition.\" arXiv preprint arXiv:1804.03209 (2018).", "claims_and_evidence": "The claims are supported by comparative experiments, theoretical convergence proofs, and ablation studies.", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria are sound, with the authors using various benchmark datasets (CIFAR-10, CIFAR-100, ImageNet, DVS-CIFAR10) that test their BSO and T-BSO algorithm.", "theoretical_claims": "I carefully reviewed the convergence proofs in Appendix A (pages 11-12).", "experimental_designs_or_analyses": "I carefully examined the experimental design and found it to be generally sound.", "relation_to_broader_scientific_literature": "The paper makes contributions to binary spiking neural network research by proposing an online training algorithm that reduces memory overhead.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "Strengths:\n1. The paper presents a novel approach to training BSNNs by integrating online training with a unique gradient momentum-based weight-flipping mechanism, which is a promising solution to existing memory and computational constraints.\n2. The authors provide comprehensive theoretical analysis, including detailed convergence proofs for their proposed BSO and T-BSO algorithms.\n\nWeaknesses & Questions:\n1.  The T-BSO may introduce some additional computational complexity compared to the base BSO method.\n2. While promising, the results are primarily demonstrated on image classification tasks, and the method's performance on other domains remains to be explored.\n3. The proposed method introduces additional computational complexity through momentum-based weight flipping and adaptive thresholding.\n4. As a efficiency-focused work, why the authors did not compare efficiency metrics such as training or inference time, OP, SOP, or energy consumption.\n\n[1] Bitsnns: Revisiting energy-efficient spiking neural networks\n[2] Towards energy efficient spiking neural networks: An unstructured pruning framework\n[3] Towards Accurate Binary Spiking Neural Networks: Learning with Adaptive Gradient Modulation Mechanism", "other_comments_or_suggestions": "It is recommended that the author conduct more thorough experiments to verify the effectiveness and efficiency of the method from multiple perspectives.", "questions_for_authors": "No.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jzotxS5EHa", "reviewer": "Reviewer_AqeW", "summary": "This paper introduces the Binary Spiking Online Optimization (BSO) algorithm, designed to reduce memory overhead in training Binary Spiking Neural Networks (BSNNs) by eliminating latent weight storage and making memory requirements time-independent. It also presents T-BSO, a temporal-aware variant that adjusts thresholds dynamically using gradient information across time steps for improved optimization.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Response to W1:\"T-BSO introduces more computational overhead due to additional gradient computations.\"**\n\nAs noted in our Response to Reviewer Q2xs (Response to W1), we have discussed that T-BSO incurs additional computational overhead due to the necessity for extra gradient computations. We kindly direct you to that section for a comprehensive explanation.\n\n**Response to W2:\"The algorithms require careful tuning of hyperparameters, which can be time-consuming.\"**\n\nRegarding hyperparameter tuning, we agree that the method introduces several new parameters (e.g., momentum decay factor, threshold). That said, we found the method to be **relatively robust** to a wide range of settings, and we provide the default configurations in the supplementary material to facilitate reproducibility. We provide an ablation experiment on hyperparameters in the table below, showing that the convergence of T-BSO and BSO is not sensitive to hyperparameters. The base setting is$(\\gamma = 1 \\times 10^{-6}, \\beta_1 = 1 \\times 10^{-3}, \\beta_2 = 1 \\times 10^{-5})$. As observed from the table, the final convergence result is not significantly affected by variations in hyperparameters, provided they remain within a certain range.\n\n| Model | Base | $\\gamma $ |$\\beta_1$   | $\\beta_2$ |\n|:-----------:|:-------:|:--------------:|:----------------------:|:------------------------:|\n| T-BSO | 75.72 |`75.68(5e-7)`, `74.68(2e-6)` | `74.95(5e-4)`,`75.36(2e-3)` | `75.10(5e-6)`,`75.52(2e-5)` |\n\n**Response to Q1:\"The author mentioned that BSO is based on a direct learning method. Why not consider the conversion-based method, but choose the direct training method?\"**\n\nWe opt for direct training over ANN-to-SNN conversion primarily due to its superior compatibility with online training and streaming data scenarios. While conversion-based methods are effective in static image classification tasks, they generally lack the ability to **learn continuously**, rendering them less suitable for online and event-driven environments. In contrast, direct training enables frame-by-frame updates and localized gradient propagation.\n\n**Response to Q2:\"The author points out in the introduction, 'leverage the substantial efficiency advantages in BSNN training'. Specifically, how does the BSO algorithm demonstrate its advantages in BSNN?\"**\n\nThe BSO algorithm directly associates the update of binary weights with the sign flip induced when the momentum of the gradient exceeds a threshold. This eliminates the need for latent weights, and this flip mechanism is theoretically more energy-efficient.\n\n**Response to Q3:\"Does the introduction of adaptive thresholds in the T-BSO version of BSO affect the neural dynamics of the online learning framework?\"**\n\nWe designed the adaptive threshold mechanism in T-BSO to be biologically plausible and hardware-friendly, updating thresholds based on the momentum of temporal gradient. While it alters the flip condition of weights, it does not fundamentally change the underlying online training framework or disrupt spiking dynamics. Instead, it allows the network to **dynamically adjust** its sensitivity to incoming signals, enhancing robustness and efficiency. We also observed that adaptive thresholds help stabilize training and improve generalization across varying input distributions.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "Yes, I have checked the proof and theorems.", "experimental_designs_or_analyses": "Yes", "relation_to_broader_scientific_literature": "This article proposes the BSO algorithm and its variant T-BSO, further promoting edge intelligent computing.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "Strengths:\nBSO reduces memory overhead by eliminating latent weight storage, making it ideal for resource-constrained environments; Experimental results show better accuracy and efficiency than existing methods.\nWeakness:\nT-BSO introduces more computational overhead due to additional gradient computations. \nThe algorithms require careful tuning of hyperparameters, which can be time-consuming.", "other_comments_or_suggestions": "No", "questions_for_authors": "1. The author mentioned that BSO is based on a direct learning method. Why not consider the conversion-based method, but choose the direct training method?\n2. The author points out in the introduction, 'leverage the substantial efficiency advantages in BSNN training'. Specifically, how does the BSO algorithm demonstrate its advantages in BSNN?\n3. Does the introduction of adaptive thresholds in the T-BSO version of BSO affect the neural dynamics of the online learning framework?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7MoaZ8juma", "reviewer": "Reviewer_CwUQ", "summary": "The paper introduces the Binary Spiking Online Optimization (BSO) algorithm, an approach to training BSNNs that reduces training memory overhead while maintaining computational efficiency. The basic BSO eliminates latent weight storage and uses momentum-based gradient accumulation to generate weight-flipping signals, while the temporal-aware T-BSO variant captures gradient information across time steps with an adaptive threshold mechanism. Extensive experimental validation validates the effectiveness of BSO and T-BSO.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Response to Q1:\"The paper lacks analysis of how the proposed methods would perform with more complex network architectures beyond standard VGG and ResNet models, such as transformer-based architectures or recurrent networks.\"**\n\nWe agree that evaluating BSO and T-BSO on more advanced architectures would strengthen our claims. While we focus on VGG and ResNet for their interpretability and popularity in the BSNN literature, our method is **model-agnostic**. This makes it naturally extendable to hierarchical and attention-based architectures such as spiking Transformers or recurrent SNNs. We have discussed the Spike-Former results in our response to Reviewer Tbrf (Response to W2) and kindly refer you to that section for transformer-based results. In a similar vein, we explored the FeedBack (recurrent) SNN [1] and employed VGG11 with T-BSO, training for 300 epochs on CIFAR-10, achieving an accuracy of 92.43%. This demonstrates the versatility of our method and its ability to extend naturally to **different network architectures**.\n\n**Response to Q2:\"Do BSO and T-BSO lead to reductions in operations (OPs/SOPs) and energy consumption?\"**\n\nAs mentioned in our response to Reviewer Q2xs (Response to W4), we have explained that BSO and T-BSO result in reduced operations and energy consumption due to their lower firing rates. We kindly refer you to that section for a detailed explanation.\n\n**Response to Q3:\"The authors only validated the proposed methods on simple image classification tasks. Can they be extended to other applications\"**\n\nOur initial experiments focused on classification as a standard benchmark for spiking models. However, the proposed method is task-agnostic and can be integrated into other applications. As stated in our response to Reviewer Q2xs (Response to W2), we conducted experiments on the GSC dataset, which demonstrates the effectiveness of our method across different tasks.\n\n**Response to Q4:\"There's no analysis of BSO's convergence behavior under noisy data, which would be valuable for real-world applications where data quality varies.\"**\n\nThis is an important consideration for real-world deployment. The update mechanism of BSO, which relies on the momentum of the gradient exceeding a threshold to induce a sign flip, inherently offers robustness against noise in the data. This is due to the thresholding operation, which acts as a filtering mechanism, allowing only **significant gradient information** to influence the weight updates. As a result, smaller fluctuations in the gradient caused by noise are less likely to trigger updates, making the algorithm less sensitive to noisy data. The training methods of BSO and T-BSO are not only resilient to gradient noise fluctuations but also to the noise in the input data. To validate this, we conduct experiments using the CIFAR-10, CIFAR-100, and VGG-11 architectures, where three different types of Gaussian noise are added to the input images during testing. The performance of BSO and T-BSO is then compared in terms of accuracy. The experimental results demonstrate that both BSO and T-BSO exhibit a certain level of noise robustness, with T-BSO outperforming BSO under identical noise conditions.\n\n| Dataset   | Algorithm | Base  | $(\\mu=0,\\sigma=0.1)$ | $(\\mu=0,\\sigma=0.4)$ | $(\\mu=0.3,\\sigma=0.1)$ |\n|----------|-------|-------|----------------------|----------------------|------------------------|\n| **CIFAR-100** | BSO   | 68.73 | 65.82 (-2.91)        | 46.42 (-22.31)       | 64.95 (-3.78)          |\n|          | T-BSO | 74.17 | 70.34 (-3.83)        | 51.21 (-22.96)       | 69.45 (-4.72)          |\n| **CIFAR-10**  | BSO   | 93.30 | 91.02 (-2.28)        | 80.61 (-12.69)       | 92.75 (-0.55)          |\n|          | T-BSO | 94.32 | 93.01 (-1.31)        | 85.47 (-8.85)        | 93.12 (-1.20)          |\n\n**Response to Q5:\"There is significant room for improvement in the writing of the paper.\"**\n\nThank you for your candid feedback. We sincerely apologize for our poor writing. We will carefully revise the manuscript to improve clarity, coherence, and presentation. This includes refining the descriptions of our methods, improving figure quality throughout the paper.\n\n[1] Xiao, Mingqing, et al. \"Training feedback spiking neural networks by implicit differentiation on the equilibrium state.\" Advances in neural information processing systems 34 (2021): 14516-14528.", "claims_and_evidence": "Yes, most of the claims are supported by appropriate evidence.", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria in the BSO paper are generally well-aligned with the problem they aim to address.", "theoretical_claims": "I examined the theoretical claims and proofs presented in the paper, focusing on the convergence guarantees for both BSO and T-BSO algorithms. The theoretical analysis is mathematically sound.", "experimental_designs_or_analyses": "I examined the experimental designs and analyses presented in the paper, focusing on the validation of the BSO and T-BSO algorithms across different datasets and comparisons with baseline methods. (1) The authors compare BSO and T-BSO against both online training methods (OTTT, NDOT, SLTT) and BSNN methods (Q-SNN, CBP). (2) The training paradigms are analyzed using the FF ratio and C2I ratio metrics to investigate the optimization stability.", "relation_to_broader_scientific_literature": "The BSO paper makes contributions to several domains, such as SNNs, online learning, and efficient computing. By integrating these research domains, BSO addresses the contradiction between BSNNs' inherent efficiency and their memory-intensive training processes.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "Strengths:\n1. By eliminating latent weights and making memory requirements independent of time steps, the BSO and T-BSO provide a meaningful solution for resource-constrained applications.\n2. The theoretical analysis is comprehensive. The formal regret bounds provide mathematical guarantees for convergence.\n3. The ablation studies illustrate the effectiveness of the methods, particularly the role of momentum in stabilizing training for binary networks.\n\nWeaknesses & Questions:\n1.The paper lacks analysis of how the proposed methods would perform with more complex network architectures beyond standard VGG and ResNet models, such as transformer-based architectures or recurrent networks.\n2.Do BSO and T-BSO lead to reductions in operations (OPs/SOPs) and energy consumption? These metrics are crucial for edge deployment scenarios.\n3.The authors only validated the proposed methods on simple image classification tasks. Can they be extended to other applications?\n4.There's no analysis of BSO's convergence behavior under noisy data, which would be valuable for real-world applications where data quality varies.\n5.There is significant room for improvement in the writing of the paper.", "other_comments_or_suggestions": "No.", "questions_for_authors": "See Weaknesses & Questions.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper develops online training of Binary Spiking Neural Networks (BSNNs) and proposes Binary Spiking Online Optimization (BSO), that significantly
reduces training memory while maintaining energy efficiency during the forward pass. Moreover, a temporal-aware variant T-BSO is introduced.
Theoretical analysis shows that the convergence of both BSO and T-BSO algorithms based on the regret bounds. Experiments on datasets like CIFAR-10, CIFAR-100, ImageNet, and DVS-CIFAR10, verify its effectiveness.

The main strengths of this paper are:

- the potential to advance resource-efficient neuromorphic computing
- a novel approach to training BSNNs online with theoretical guarantee
- experiments show the proposed method achieve better accuracy and efficiency than existing methods.

The main weaknesses of this paper are:

- the needs of more comparison with other methods (Reviewer Tbrf, AqeW)
- the needs of more discussion like spiking Transformers and more complex architectures (Reviewer Tbrf, CwUQ), efficiency metrics (Reviewer Q2xs, AqeW, CwUQ), computational complexity (Reviewer Q2xs), convergence behavior under noisy data (Reviewer CwUQ)

During the rebuttal phase, all reviewers acknowledged.  Overall, this paper received a consistent positive recommendation, 4 accept from the reviewers.  All the concerns raised during review were  addressed in the later rebuttal. 
After reading all the review, the discussion and the rebuttal, the AC agrees its novelty, its potential and superior performance。 In this case, the AC recommends an Accept.
Decision: Accept (poster)