Online Stabilization of Spiking Neural Networks
Authors: Yaoyu Zhu
Jianhao Ding
Tiejun Huang
Xiaodong Xie
Zhaofei Yu
Venue: ICLR 2024 spotlight (2024)
Keywords: spiking neural networks
online training
Primary area: applications to neuroscience & cognitive science

Abstract: Spiking neural networks (SNNs), attributed to the binary, event-driven nature of spikes, possess heightened biological plausibility and enhanced energy efficiency on neuromorphic hardware compared to analog neural networks (ANNs). Mainstream SNN training schemes apply backpropagation-through-time (BPTT) with surrogate gradients to replace the non-differentiable spike emitting process during backpropagation. While achieving competitive performance, the requirement for storing intermediate information at all time-steps incurs higher memory consumption and fails to fulfill the online property crucial to biological brains. 
Our work focuses on online training techniques, aiming for memory efficiency while preserving biological plausibility. 
The limitation of not having access to future information in early time steps in online training has constrained previous efforts to incorporate advantageous modules such as batch normalization. 
To address this problem, we propose Online Spiking Renormalization (OSR) to ensure consistent parameters between testing and training, and Online Threshold Stabilizer (OTS) to stabilize neuron firing rates across time steps. Furthermore, we design a novel online approach to compute the sample mean and variance over time for OSR. Experiments conducted on various datasets demonstrate the proposed method's superior performance among SNN online training algorithms.
Our code is available at https://github.com/zhuyaoyu/SNN-online-normalization.

Published as a conference paper ICLR 2024
ONLINE STABILIZATION OF SPIKING NEURAL NETWORKS
Yaoyu Zhu's Jianhao Ding', Tiejun Huang! 2, Xiaodong Xie' & Zhaofei Yu 12
School of Computer Science, Peking University Institute for Artificial Intelligence, Peking University
ABSTRACT
Spiking neural networks (SNNs), attributed to the binary, event-driven nature of spikes, possess heightened biological plausibility and enhanced energy efficiency on neuromorphic hardware compared to analog neural networks (ANNs). Mainstream SNN training schemes apply backpropagatio--troug---t-t
(BPTT) with surrogate gradients replace the non- differentiable spike emitting process during backpropagation. While achieving competitive performance, the requirement for storing intermediate information at all time-steps incurs higher memory consumption and fails to fulfill the online property crucial to biological brains. Our work focuses on online training techniques, aiming for memory efficiency while preserving biological plausibility. The limitation of not having access to future information in early time steps online training has constrained previous efforts to incorporate advantageous modules such as batch normalization. To address this problem, we propose Online Spiking Renormalization (OSR) to ensure consistent parameters between testing and training, and Online Threshold Stabilizer (OTS) to stabilize neuron firing rates across time steps. Furthermore, we design a novel online approach to compute the sample mean and variance
time for OSR. Experiments conducted various datasets demonstrate the proposed method's superior performance among SNN online training algorithms. Our code is available at https///ttthbbbccmmm/yyyyyuu///o normalization
1 INTRODUCTION
Regarded as the third generation of neural networks, spiking neural networks (SNNs) possess a greater level of biological plausibility (Zenke al. [202]] than their second generation counterparts analog neural networks (ANNs) due to the binary and event-driven nature of spikes. The binary nature of spikes in SNNs eliminates the need for multiplication during inference, leading to improved energy efficiency when deployed on neuromorphic hardware (Furber et al., 2014: Merolla et al. [2014] Shen et al. [016)] Davies et al. 2018 Pei et al. 2019). However, the discontinuity of binary spikes also poses challenges in the training of SNNs.
To address the non -on-differentiable issue associated with the spike emitting process in SNN training. various approaches have been proposed. The mainstream direct training techniques use surrogate gradients to address this problem, which replaces the non-differentiable Heaviside function during the spike firing process with a differentiable surrogate function (Neftci et al. 2019). In addition to this, they just regard SNNs as binary recurrent neural networks (RNNs) and use cocckoopaggtiion-troogg---oo
(BPTT) for SNN training (Bellec) al. 2018 Zenke& & Ganguli 2018 Wu et al. | 20188). 8). Although competitive performances are achieved on the CIFAR- 10/ 100 and ImageNet datasets (Deng et al. [022 Fang et al., (202)) with a relatively short simulation time, these methods require storing intermediate information of all time-steps for gradient backpropagation. An alternative approach to train SNNs is use the assistance of ANNs. Several works first train ANNs and
*Corresponding author: wuzf12@pku.edu.cn

---

Published as a conference paper ICLR 2024
then convert them to SNNs (Cao al. 2013 Rueckauer al. 2017 Han al. 2020 Bu al. 2022a Deng & Gu 2021 Bu et al. 2022b). However, these methods often require a longer simulation time and result in more fired spikes. The long simulation time will lead to high latency, while more fired spikes will consume more energy. Overall, these approaches bring about extra expenses either in the training phases or in the lesting phases while not satisfying the online property of the learning process in biological brains.
Recently, online training techniques have been developed to save memory costs while maintaining the biologically plausible online property during the training process. However, the limitation of not having access to future information in the early time steps has constrained previous efforts to incorporate advantageous modules such as batch normalization (BN). In this work, we design a mechanism that bypasses the need for future information while maintaining consistency across time-steps, thereby reducing the overfitting problem associated with treating different time-steps with different BN. Our main contributions can summarized as follows:
1. We propose Online Spiking Renormalization (OSR), ensuring consistent scale and shift parameters between testing and training. This helps eliminate the normalization parameter difference when applying BN separately for each time-step. In addition, we introduce an online approach for computing a variable's 's all-time mean and variance that dynamically changes over time for OSR.
2. We devise Online Threshold Stabilizer (OTS), aiming at stabilizing neuron firing rates across varying time steps, which also effectively regulates the overall firing rate.
3. We conduct experiments CIFAR1O, CIFARIO0, CIFARIO-DVS Wys-Gestr and Imagenet datasets and demonstrate that our proposed method achieves ofate-oftthe-ae art performance among SNN online training algorithms.
2 RELATED WORK
2.1 ONLINE TRAINING APPROACHES
Online training allows real-time parameter updates as new data arrives, especially useful for RNNs and SNNs spanning multiple time-steps. This mechanism serves curtail memory usage, a particularly advantageous feature when dealing with many time-steps.
Existing literature RNNs has delved into various approaches to online learning. Real-time recurrent learning (RTRL), introduced by Williams & Zipser (189),, propagates partial derivatives of hidden states across parameters throughout time, enabling the computation of gradients in forward-in-time manner. Many recent research endeavors, exemplified by UORO (Tallec & Ollivier 2017). KF-RTRL (Mujika et al.) 2018), and SnAp (Menick et al. 2020), have explored enhancing the memory and time efficiency of RTRL through tailored approximations for more pragmatic utilization. Another work put forward a proposition to update parameters in an online fashion, utilizing decoupled gradients coupled with regularization each time-step (Kag & Saligrama, 202T).
the domain of SNNs, numerous studies have drawn inspiration from online training techniques developed for RNNs. Some of these works adopt the fundamental principles of RTRL and tailor them to streamline the training process for SNNs (Zenke& & Ganguli 2018: Bellec et al.) 2020; [Bohnstingl et al. (022). Yin
et (2022) directly applied the approach proposed by Kag & Saligrama| (2021) to train SNNs. Zenke &
Ganguli (2018) connected the online learning rule for leaky integrate-and
(LIF) neurons with the nonlinear Hebbian three-factor rule, and Kaiser et al. (2020) extended the neuron model to exponenaxponetiall spike- response model. Xiao et al. (2022) successfully extended online training methodologies to accommodate large-scale tasks such the ImageNet classification. However, all these works did not consider incorporating network modules like batch normalization to enhance the network performance. As a result, they suffer from a performance disadvantage compared to their BPTT counterparts.
2

---

Published as a conference paper ICLR 2024
2.2 NORMALIZATION MECHANISMS
Normalization mechanisms are commonly used in neural networks to stabilize network training, which speeds up convergence and enhances network performance. Typical normalization techniques include batch normalization (BN) (Toffe& & Szegedy 2015), instance normalization (IN) (Ulyanov et al. 2016), group normalization (GN) (Wu & He| | 2018), and layer normalization (LN) (Ba et al. 2016)) A subsequent work, batch renormalization (loffe| 2017). improves BN by eliminating the difference between the batch mean and variance between the training and testing phases.
In spiking neural networks, researchers have also tried to incorporate normalization techniques to enhance SNN performance. For instance. Kim & Panda (202T) proposed BNTT to regulate firing rates by utilizing separate BN parameters at different time steps. Zheng et al.) (202T) proposed threshold-dependen batch normalization (tdBN), which extends the scope of BN to the additional temporal dimension and takes into account the impact of threshold on firing rates. TEBN (Duan et al. 2022) combined elements from both of these approaches by applying BN across the shift parameters at different time steps. Although most works apply normalization to the input current, some studies explore normalization for other variables. For example, PSP-BN (Ikegawa et al. (022) used unique statistics, the second raw moment of post-synaptic potential, as the denominator for normalization, which can be inserted right after the spiking functions. This approach leads to higher complexity of BN parameters and the potential risk of breaking the temporal coherence of information. Among the aforementioned works, the most successful ones (Duan et al. 2022 Zheng et al. 202I) used information from all time-steps for BN. However, these methods cannot be directly applied to online learning.
dimension, while utilizing separate scale and
3 PRELIMINARIES
3.1 LEAKY INTEGRATE AND FIRE NEURON
Spiking neurons are the basic building blocks of SNNs, with the LIF neuron model being the most commonly used (Gerstner et al. 2014). The dynamic of the LIF neuron before firing can be described by:
du(t) dt (u(t) usea) + +(t),
(I)
where u(t is the membrane potential of the neuron at time t. I(t) is the imput current received by the neuron, 7 is the membrane time constant, and uyet is the resting potential. When membrane potential u(t) reaches a certain threshold 0, the neuron will emit a spike, and ut)) will be suddenly reset to a value Ureset- In practice,
we often use discrete form of Eq. 1 which can be represented as:
u' [t] =(1 - )u' - 0.5] W's s')'
(2)
s [4] 0 0),
(3)
n ( + 0.5] = (1-
(4)
Here, we use a tensor form that u', W and sx denote the membrane potential tensor, weight matrix between layers 1- and output spike tensor of layer I, respectively. Among them, v²(1] is the membrane potential after decay and adding input but before the reset, and u' [ _ is the membrane potential after reset. θ is Heaviside step function. The element of s [t]] equals 1 if the neuron fires and 0 otherwise.
4
METHODS
Our holistic method is illustrated in Figure I In the following parts, we first briefly introduce the forward and backward propagation processes of our algorithm and then elaborate on the modules we add to the network.
3

---

Published as conference paper ICLR 2024
Forward
*...
Backward
1
2" 1d
2: 10
1-2
BIN
BN
BNN
OSR
OSR
OSR
LIF
LIF
LIF
LIF
LIF
LIF
LIF
LIF
LIF
OT5
GTS
OTS
Layer Index
Layer Index
Layer Indox
10
Surrogate learning
Online learning OTTT
Online learning This work
BPTT
Online Spiking Renoraliiation (OSR)
Online Threshold Stabilizer (OTS)
for training
U
Illustration online stabilization techniques for SNN. Our method uses online spiking renormal- ization to improve the generalization and an online threshold stabilizer to regulate the firing rate within the framework of online SNN training, which requires less memory usage than BPTT training. OTTT (Xiao et al. 2022 adopts normalization-fie networks and thus has BN modules.
Figure 1:
The major modules are online spiking renormalization (OSR) and online threshold stabilizer (OTS). Besides, an online calculation method of all-time mean and variance introduced in OSR.
4.1 FORWARD AND BACKWARD PROPAGATION
In the forward stage, our method uses the LIF formulas (Eqs. 2 4). The OSR replaces [' with over time-steps.
renorm(s'-(())) in Eq. 2 and the OTS changes threshold Eq.
In the backward stage, we select the TET loss (Deng et al. 2021 as our loss function since the loss function needs provide feedback at each time-step:
1
T Σ t=1
-=
Σ C. 1=1
Σ
(1-9 i=1
De log 0f(1] + > (o.lt) (v:)))
(5)
T
where T the total simulation time, Y; denotes whether the label equal to i, and o is the spikes of output neuron time in the output layer. An additional MSE loss introduced regularization term (as proposed by Deng et al. (2021)) with weight e, and @(ys) is the target value of y set MSE loss.
For the online gradient propagation, and reset (from the next time step to the last time step) for membrane potential (as shown in Figure II. Then the gradients received by membrane potential and weights become:
remove the propagation path of neuron membrane potential decay
dc
dc,
ds Tu
(6)
0ul, 1]
1
2C
ac, as add
(7)
Ow! zz
Lsy 0ull Sw
4

---

Published as a conference paper ICLR 2024
Note that Eq. 7 just sums up the derivative of Ls to why all time-steps. there is backward forward
temporal dependency both in Eqs. 6 and
4.2
INCORPORATING BATCH NORMALIZATION INTO ONLINE ALGORITTMMS
Zheng et al. (2021), : Kim Panda (2021) Duan et al. (2022). it is shown that applying BN once across all time-steps, rather than separately on each time-step, yields superior performance. However, in online training. it is impractical as we need normalization before having information on all time-steps. As per Duan et al. (2022), using mean and variance across all time-steps is crucial for reducing temporal covariate shift and enhancing performance. key feature of normalizing by the global mean and variance is that, the transformation of all tme-steps the same during normalization. Therefore, question naturally arises: we normalize inputs at all time-steps with the same mean and variance when we do not have the all-tine all-time data?
Online Spiking Renormalization (OSR). Although do not have the whole data of the current batch, have data from previous batches and can apply BN transformation at all time-steps based on these data. The running mean j and running variance 6² is good choice. Using them as the normalization parameter brings an additional benefit: The BN transformation will be the same between the training stage and the inference stage. Specifically, we apply the transformation
1 f
||t] 7
+
(8)
0
during the forward stage training, where [[]] is the neurons' input currents to be normalized. The next question is: How to compute gradients in the backward stage if we use this forward transformation? The j and 6² come from previous data instead of the current batch data. Therefore, if no additional mechanisms involved, this 'standardization' just plays the role of linear transformation instead of real normalization. Our solution is online spiking renormalization (OSR), which first applies real normalization and then unifies transformation among time-steps steps by another linear transform. To be specific, we first normalize [[t] to
[(t]
%((=e()) and then linearly transform it twice ][t] 7 [o
+ B::
Vo+te
11
Vo2 +
M 14-
10
i
B 7
[
+ B.
(9)
V +E
V0
+E
Eq. 9 denotes the normalization followed by linear transformation. The gradients for I I(t), 7, are:
dc dc
af[t] Vo
(10)
[î
0c
Σ OC i vollth
plt] - i
(11)
0y
[4,
Vo2 +e Vo +62+
0C
OL
Σ
(12)
08
Online Calculation of All-time Mean and Variance. In OSR, the running mean K and rumning variance ,² are the running average of all-time mean μ and variance o² of batch. To keep the memory cost low, we need to calculate these all-time statistics in an online fashion, utilizing the mean and variance of each time-step: =I1),... [T[] and o [1].).. . [T) Their relationship can be described by the following equations:
LtC.
0² [t]
(L m(tl)
(13)
5

---

Published as a conference paper ICLR 2024
T T Σ
(14)
mT
(m
mml
t=1
m70
Σ ult 1²2
(15)
Hence, we can initialize and x² as O for each batch, add + su(t) to μ and add ( + m(t)?) to o² at each time step, and subtract μ² from o² at the last time step.
Online Threshold Stabilizer (OTS). To enhance the stability of mean and variance in the OSR process during training, we introduce the OTS mechanism. The variable subject to normalization is the input current of neurons, and our objective is to ensure the mean and variance of it remain stable across all time-steps. This raises a question: When should we intervene to stabilize the mean and variance of input currents?
The mean and variance of the input current in layer are significantly influenced by the output spikes from the preceding layer, making it essential to stabilize the firing rate of each layer. The firing rate is determined by the proportion of membrane potential surpassing the firing threshold within discrete stees. Consequently, we can adjust either the membrane potential or the firing threshold to regulate the firing rate. Between these options, regulating the firing threshold stands out as a judicious choice: leaves the neuronal dynamics unchanged and only impacts backward propagation by altering the values of the surrogate function.
Specifically, we assume the membrane potential of neurons one layer at time f follows a normal distribution N (Mmem [,], mem [t] (where we denote O(t), [mmm [+], and mem are the threshold, mean of membrane potential, and variance membrane potential at time t), then the firing rate this layer time is
--1 0-1(0
(Olt) O [4]
#mom [f]
(16)
where ((x)
1 S e 2 dy is the cumulative distribution function of normal distribution. To keep this V2n
ratio constant among time-steps, we need keep the quantile
[[t]-Humm[] constant. Under this control, the [aeu
adjusted threshold at time t, 8 = /mmm [t] + m [t)
0[1]-
The overall algorithm description is provided in Appendix B
4.3 THEORETICAL ANALYSIS
this section, we discuss how our online threshold stabilizer (OTS) helps stabilize online spiking renormal- ization (OSR). The process involves three stages: adjusting the firing threshold of layer / - 1. corresponding adjustment of the firing rate of layer / 1, and regulating the mean and variance before normalization in layer to ensure stability. This involves two crucial aspects: adjusting the threshold for stable firing rate, which subsequently stabilizes the mean and variance. For the step from threshold to firing rate, existing research has shown that when a LIF neuron receives constant input with Gaussian noise, the membrane potential will have a Gaussian distribution (Hohn & Burkitt (000T). This implies the reasonableness of our Gaussian distribution assumption of membrane potential in OTS, further supporting its process. For the step from firing rate mean and variance, studying the property of the all-time sample variance o² is a good choice: In Eq. 15 g² can be split into two parts: The first part is T ΣT [²[(], which stands for the average variance inside each time-step. The second part is $ ΣT= /u|t2 - 2², which is the variance of the mean at each time-step (variance of [I[),.... [T))). To stabilize the whole training process, time-steps to vary as little as possible. In other words, we want the variance of the mean among time-steps to
want the mean among different
be low. On the other hand, for variance inside each time-step, we do not need it to be low.
6

---

Published as a conference paper ICLR 2024
Denote p(t] to be the firing probability of each neuron time-step t and gross firing probability p 7 Ei= pft). To proceed with the theoretical derivation, we must establish the following assumptions:
Assumption 4.1. Assume all entries of [t) (of size B Cia) and W (of size Cin Cout) independent for st S T. all [t) obey i.i.d Bernoull(pftt) distribution, and all w/, obey any i.id distribution.
Under the above assumptions, we have the following conclusions (note we only discuss the expectation of the target variables since both the sample mean μ and the sample variance o² are estimated statistics):
Theorem 4.2. When Assumption 4.7 holds and the gross firing rate p holds constant, then the expectation of sample variance of ju[t] among time time-steps E
uutt? n²] increases when the variance of firing rate among time-steps + +E=1 -1² increases.
Theorem 4.3. When Assumption 4.1 holds and the gross firing rate p keeps constant, then the expectation of variance within time-steps EE
Dim 1
[1]
keeps constant.
The detailed proof is provided in Appendix A These results indicate that given the gross firing rate (p) constant, reducing the variance of firing probability (pit)) among time-steps will reduce the variance of the mean (n(t)) among time-steps (Theorem. 4.2) but will not affect the variance inside time-steps (2 (() (Theorem. 4.3). Thus, steady firing rate helps stabilize the sample mean, which further indicates that our OTS mechanism helps our OSR mechanism. Related experimental results are shown in the ablation study.
5 EXPERIMENTS
To show the effectiveness of our proposed method, we conduct experiments on CIFARIO, CIFARIO0 (Krizhevsky et al. 2009), DVS-Gesture (Amir et al. 2017). CIFARIO-DVS (Li et al. 2017). and Imagenet Deng et al.) 2009) datasets to evaluate the performance of our method. The model we choose is consistent with OTTT (Xiao et al., 2022 to conduct fair comparison. All experiments run Nvidia RTX 4090
GPUs with Pytorch 2.0. The implementation details are provided in Appendix C
5.1 COMPARISON WITH OTHER WORKS
Here we compare our approach with previous SNN training methods. We select the BPTT-based algorithms tdBN (Zheng et al., 2021), D SEW (Fang al. 2021 TET (Deng et al. 2021). TEBN (Duan et al., 2022). and an online algorithm OTTT (Xiao et al. (022). The results have shown that our algorithm performs well on all datasets. For the CIFARTO TO dataset, we have outperformed tdBN, OTTT, and TET. For the CIFAR 100 dataset, we have outperformed TET and OTTT. For the listed methods, including tdBN and OTTT. For the CIFARIO-DVS dataset, we have outperformed tdBN and OTTT. For the Imagenet dataset, we have outperformed tdBN and OTTT. Note that the network that OTTT uses (NF-Resnet-34) adds membrane potential in the shorteut connection, which enhances overall performance over SEW-Resnet-34 34 and Resnet-34. We test our method for the same architecture (the last line) and achieve better performance with fewer time-steps. Among the online algorithms, we have outperformed OTTT on all datasets with fewer time-steps (T = 4 vs T 6). In addition, although the overall performance offte-of-the-arr based algorithms outperforms the online ones in Table I they require more memory, especially when the number of total time steps is large (the detailed information is provided in Section 5.2)
dataset, we have outperformed all
Comparison with Vanilla BN. To show the necessity of our approach, we compare it with a vanilla BN, which applies BN each time-step solely based data from that time-step. The result shown in the BN (vanilla) line for the Imagenet dataset, and we can see our approach outperforms this vanilla BN by around 4%. More detailed ablation studies for OSR and OTS various datasets are provided in Appendix D
7

---

Published as conference paper ICLR 2024
Table Performance comparison on CIFAR-10/100, DVS Gesture, CIFARIO-DVS, and Imagenet
Dataset
Model
Online
Architecture
Time steps
Accuracy
WBB
Zheng et al.
2021
Resnet-19
4
92.92%
TET
Deng et al.
2021
Resnet-19
94.44%
TEBN
Duan
2022
Resnet-19
4
95.58%
CIFARIO
OTTT
Xiao et al.
2022
VGGSNN
93.58% 6
VGGSNN
94.35%
Ours
Resnet- 19
95.20% 4
TET Deng et al. 2021
Resnet-19
74.47% 4
TEBN
Duan et al.
2022
Resnet-19
78.71%
CIFARI00
OTTT
Xiao al.
2022
VGGSNN
71.11% 6
VGGSNN
76.48% 4
Ours
Resnet-19
77.86%
WBN
Zheng
2021
Resnet-17
96.88% 40
DVS-Gesture
OTTT
Xiao et al.
2022
VGGSNN
96.88% 20
Ours
VGGSNN
97.57% 20
IdBN
Zheng
2021
Resnet-19
67.80% 10
TET
Deng et al.
2021
VGG-11
83.17% 10
CIFARIO-DVS
TEBN
Duan et
2022
VGGSNN
84.90% 10
OTTT
Xiao al.
2022
VGGSNN
76.30% 10
Ours
VGGSNN
82.40% 10
wdN
Zheng et
2021
Resnet-34
63.72% 4
SEW
Fang al.
2021
SEW-Resnet-34
67.04%
TET
Deng ct al.
2021
SEW-Resnet-34
68.00% 4
TEBN
Duan et
2022
X
SEW-Resnet-34
68.28%
OTTT
Xiao al.
2022
NF-Resnet-34
6
65.15%
Imagenet
BN(Vanilla)
SEW-Resnet-34
4
60.48%
OOSS
SEW Resnet-34
64.14%
ONTOSR+OTS)
NF-Resnet-34
67.54%
TTo keep consistent with OTTT, we use the name 'NF-Resnet' here represent adding membrane potential in the shortcut connection in Resnet. Note that 'NF" in OTTT stands for normalizer- but the corresponding part (weight standardization and scaling factors a, 3 along with the corresponding operations keep variance stable) of this network is climinated in work.
5.2 QUALITATIVE RESULTS
A. Memory Usage: We compare the training memory usage between online algorithms and BPTT algorithms
here. We test the case where T 2, 4, 6, 8, 10, 15, 20, 25, on the CIFARIO dataset with VGGSNN architecture and a batch size of 128. The memory usage statistics are plotted in Figure 2 (a). We can see that our method maintains a constant memory requirement irrespective of time-steps, whereas BPTT approaches scale memory usage linearly with the number of time-steps. In addition, even when the number of time-steps is as low as 2, the memory cost our algorithm is still lower than that of its BPTT counterpart.
We compare the firing rate statistics among different configurations of our proposed modules. We test these statistics on Imagenet, using the SEW-Resnet-3e architecture with total time-steps T = 4. The gross firing rate statisties listed in Table and the per-time-step firing rates plotted in Figure (b). Results have shown that OTS successfully decreases the gross firing rate, which meets our expectations since it raises the thresholds in the latter time-steps. The effect of OSR firing rates is more
B. Firing Rate Statistics:
8

---

Published as conference paper ICLR 2024
20000
*
Online training
BPTT
Average firing rate
Memory usage
0.25
15000
0.20
10000
0.15
5000
Baseline
OSR
OTS
OTS+0SR
5
10
16
20
25
30
2
3
Number time-steps
Time-step
(a)
(b)
Figure 2: (a) Comparison of memory usage between our method and BPTT. BPTT incurs memory costs linearly proportional to time-steps, whereas our approach maintains constant memory usage regardless of time-steps. (b) Firing rate statistics of different configurations. From the figures, we know that the online threshold stabilizer indeed stabilizes the firing rate among time-steps.
Table 2: Gross firing rate
Configuration Gross firing rate
OTTT (Xiao et al. 2022)
Vanilla BN
OSR
OTS
OSR+OTS 16.06%
24%
22.39%
27.34%
19.16%
interesting: When OTS is not added, it increases the gross firing rate. However, it decreases the total firing when OTS added. For per-time-step firing rates, when the OTS mechanism fire far fewer spikes in the first time-step compared with later time-steps, while the firing rate is relatively stable from the second time-step to the last time-step. Besides, our OTS mechanism has greatly alleviated but not perfectly eliminated the firing rate variation among time-steps. It slightly over-lifts the firing rate of the first time-step, which might be caused by the firing rate distribution difference among time-steps.
added, the neurons
6 CONCLUSION AND FUTURE WORK
In this paper, we investigate online training for spiking neural networks, aiming to reduce training memory costs. We integrate essential batch normalization the online training process by introducing online spiking renormalization and online threshold stabilizers to enhance training stability. Experiments diverse datasets demonstrate the effectiveness of our proposed modules. showcasing the superior performance of our holistic approach among SNN online training algorithms. However, approach currently falls short of BPTT in performance, primarily due to the absence of inner-layer and inter-layer reverse-in-time dependencies during backpropagation. Addressing the inner-layer dependency might involve incorporating eligibility traces, but effectively managing the significant inter-layer dependency in online learning remains a challenge. Morcover, achieving biologically plausible learning necessitates local (Journé et al. 2022) and event-driven (Zhu et al. 2022) properties addition to online behavior- areas we haven' i extensively explored in this work. These shortcomings present promising avenues for future research and deeper investigation.

---

Published as a conference paper ICLR 2024
ACKNOWLEDGEMENTS
This work was supported by the National Natural Science Foundation of China(62176003,, 62088102) and by Beijing Nova Program (20230483362..
REFERENCES
Aron Amir, Brian Taba, David Berg, Timothy Melano, Jeffrey McKinstry, Carmelo Di Nolfo, Tapan Nayak, Alexander Andreopoulos, Guillaume Garreau, Marcela Mendoza, et al. A low power, fully event-based gesture recognition system. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7243-7252, 2017.
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. arXiv preprint a607: 064500 2016.
Guillaume Bellec, Darjan Salaj, Anand Subramoney, Robert Legenstein, and Wolfgang Maass. Long short- term memory and learning-to-lean in networks of spiking neurons. Advances in Neural Information Processing Systems, 31:795 805, 2018.
Guillaume Bellec, Franz Scherr, Anand Subramoney, Elias Hajek, Darjan Salaj. Robert Legenstein, and Wolfgang Maass. A solution to the learning dilemma for recurrent networks of spiking neurons, Nature Communications, 1(1):36225, December 2020. ISSN 2041-1723. doi: 10.10038/441447-00011223333
Thomas Bohnstingl, Stanislaw Wozniak, Angeliki Pantazi, and Evangelos Eleftherion. Online Spatio- Temporal Learning in Deep Neural Networks. IEEE Transactions on Newral Networks and Learning Systems, pp. 1-15, 2022. ISSN 2162-237X, 2162-2388. doi: 10. 2022.33.32...
Tong Bu, Jianhao Ding, Zhaofei Yu, and Tiejun Huang. Optimized potential initialization for low-latency spiking neural networks. In In Proceedings of the AAAI Conference on Artificial Intelligence, pp. 11-20, 2022a.
Tong Bu, Wei Fang, Jianhao Ding, PengLin Dai, Zhaofei Yu, and Tiejun Huang. Optimal ANN-SNN conversion for high-accuracy and ultra-low-latencyy spiking neural networks. In International Conference on Learning Representations, 2022b.
Yongqiang Cao, Yang Chen, and Deepak Khosla. Spiking deep convolutional neural networks for energy- efficient object recognition. International Journal of Computer Vision, 1 113(1):44-66, 2015.
Mike Davies, Narayan Srinivasa, Tsung- Han Lin, Gautham Chinya, Yongqiang Cao, Sri Harsha Cho- day, Georgios Dimou, Prasad Joshi, Nabil Imam, Shweta Jain, Yuyun Liao, Chit-Kwan Lin, Andrew Lines, Ruokun Liu, Deepak Mathaikutty, Steven McCoy, Arnab Paul, Jonathan Tse, Guruguhanathan Venkataramanan, Yi-Hsin Weng, Andreas Wild, Yoonseok Yang, and Hong Wang. Loihi: A neuromorphic manycore processor with on-chip learning. IEEE Micro, 38(1):82-99, 2018. ISSN 0272-1732. doi: 10.1109/MM.2018. 12233359..
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255. Icee, 2009.
Shikuang Deng and Shi Gu. Optimal conversion of conventional artificial neural networks spiking neural networks. In International Conference on Learning Representations, 2021.
Shikuang Deng, Yuhang Li, Shanghang Zhang, and Shi Gu. Temporal efficient training of spiking neural network via gradient weightit.. In International Conference Learning Representations, 2021.
10

---

Published as a conference paper ICLR 2024
Chaoteng Duan, Jianhao Ding, Shiyan Chen, Zhaofei Yu, and Tiejun Huang. Temporal effective batch normalization in spiking neural networks, In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances Neural Information Processing Systems, volume 35, pp. 34377-34390. Curran Associates, Inc., 2022.
Wei Fang, Zhaofei Yu, Yanqi Chen, Tiejun Huang, Timothée Masquelier, and Yonghong Tian. Deep residual learning in spiking neural networks. Advances Neural Information Processing Systems, 34:21056-21069, 2021.
Steve B Furber, Francesco Galluppi, Steve Temple, and Luis A Plana. The spinnaker project. Proceedings of the IEEE, 102(5):652 665, 2014.
Wulfram Gerstner, Werner M Kistler, Richard Naud, and Liam Paninski. Neuronal dynamics: From single neurons to networks and models of cognition. Cambridge University Press, 2014.
Bing Han, Gopalakrishnan Srinivasan, and Kaushik Roy. RMP-SNN: Residual membrane potential neuron for enabling deeper high-accuracy and low-latency spiking neural network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13558-13567, 2020.
Nicolas Hohn and Anthony N Burkitt. Shot noise in the leaky integrate-and-five neuron. Physical Review E, 63(3))031902, 2001.
Shin-ichi Ikegawa, Ryuji Saiin, Yoshihide Sawada, and Naotake Natori. Rethinking the role of normalization and residual blocks for spiking neural networks, March 2022.
Sergey loffe. Batch renormalization: Towards reducing minibatch dependence batch-normalized models. Advances in neural information processing systems, 30, 2017.
Sergey loffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning, pp. 448-444444 456.. PMLR, 2015.
Adrien Journé, Hector Garcia Rodriguez, Qinghai Guo, and Timoleon Moraitis. Hebbian deep learning without feedback. In The Eleventh International Conference Learning Representations, 2022.
Anil Kag and Venkatesh Saligrama. Training recurrent neural networks via forward propagation through time. In International Conference on Machine Learning, pp. 5189-5200. PMLR, 2021.
Jacques Kaiser, Hesham Mostafa, and Emre Neftci. Synaptic plasticity dynamics for deep continuous local learning (decolle). Frontiers in Neuroscience, 14:424, 2020.
Youngeun Kim and Priyadarshini Panda. Revisiting Batch Normalization for Training Low-Latency Deep Spiking Neural Networks From Scratch. Frontiers Neuroscience, 15:773954, December 2021. ISSN 1662-453X. doi: wttp://minnn.2..7739999999
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Hongmin Li, Hanchao Liu, Xiangyang Ji, Guoqi Li, and Luping Shi. Cifar10-dvs: an event-stream dataset for object classification. Frontiers in neuroxcience, 11 11:309, 2017.
Yuhang Li, Youngeun Kim, Hyoungseob Park, Tamar Geller, and Priyadarshini Panda. Neuromorphic data augmentation for training spiking neural networks. In European Conference on Computer Vision, pp- 631-649. Springer, 2022.
Jacob Menick, Erich Elsen, Utku Evei, Simon Osindero, Karen Simonyan, and Alex Graves. Practical real time recurrent learning with a sparse approximation. In International conference on learning representations, 2020.
11

---

Published as conference paper ICLR 2024
Paul A. Merolla, John V. Arthur, Rodrigo Alvarez-Icaza, Andrew S. Cassidy, Jun Sawada, Filipp Akopyan, Bryan L. Jackson, Nabil Imam, Chen Guo, Yutaka Nakamura, Bernard Brezzo, Ivan Vo, Steven K. Esser, Rathinakumar Appuswamy, Brian Taba, Arnon Amir, Myron D. Flickner, William Risk, Rajit Manohar, and Dharmendra S. Modha. A million spiking-neuron integrated circuit with scalable communication network and interface. Science, 345(6197):668-67)) 673, 2014.
Asier Mujika, Florian Meier, and Angelika Steger. Approximating real-time recurrent learning with random kronecker factors. Advances in Neural Information Processing Systems, 31, 2018.
Emre O. Neftci, Hesham Mostafa, and Friedemann Zenke. Surrogate gradient learning in spiking neural networks: Bringing the power of gradient-based optimization to spiking neural networks. IEEE Signal doi: 10.1 10.1109/MPP22199999933
Processing Magazine, 36(6):51-63 63, 2019. ISSN 1053-5888, 1558-0792
Jing Pei, Lei Deng, Sen Song, Mingguo Zhao, Youhui Zhang, Shuang Wu, Guanrui Wang. Zhe Zou, Zhenzhi Wu, Wei He, al. Towards artificial general intelligence with hybrid tianjic chip architecture. Nature, 572 17767):106-111, 2019.
Bodo Rueckauer, Alexandra Lungu, Yuhuang Hu, Michael Pfeiffer, and Shih-Chii Liu. Conversion of continuous-valued deep networks to efficient event-driven networks for image classification. Frontiers in neuroscience, 11:682, 2017.
Juncheng Shen, De Ma, Zonghua Gu, Ming Zhang, Xiaolei Zhu, Xiaoqiang Xu, Qi Xu, Yangjing Shen, and Gang Pan. Darwin: a neuromorphic hardware co-processor based on spiking neural networks. Science China Information Sciences, 59(2):1-5, 2016.
Corentin Tallec and Yann Ollivier. Unbiased Online Recurrent Optimization, May 2017.
Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. Instance normalization: The missing ingredient for fast stylization. arXiv preprint 160iv: 08022, 2016.
Ronald J. Williams and David Zipser. A Learning Algorithm for Continually Running Fully Recurrent Neural Networks. Neural Computation. 1(2):270-288) June 1989. ISSN 0899-7667, 1530-888X. doi: 10. 162//ecco. 1et.beec...99.11111 1989.1.2.270.
Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, and Luping Shi. Spatio-temporal backpropagation for training high-performance spiking neural networks. Frontiers in neuroscience, 12:331, 2018.
Yuxin Wu and Kaiming He. Group normalization. Proceedings of the European conference on computer vision (ECCV), pp. 3- 2018.
Mingqing Xiao, Qingyan Meng, Zongpeng Zhang, Di He, and Zhouchen Lin. Online Training Through Time for Spiking Neural Networks, October 2022.
Bojian Yin, Federico Corradi, and Sander M. Bohte. Accurate online training dynamical spiking neural networks through Forward Propagation Through Time, November 2022.
Friedemann Zenke and Surya Ganguli. SuperSpike: Supervised Learning in Multilayer Spiking Neural Networks. Neural Computation, 30(6):15141154, June 2018. ISSN 0899-7667, 1530-888X. doi: 1111://eecooaa_00110000
Friedemann Zenke, Sander M Bohté, Claudia Clopath, Iulia M Comşa, Julian Göltz, Wolfgang Maass, Timothée Masquelier, Richard Naud, Emre O Neftei, Mihai A Petrovici, et al. Visualizing joint future neuroscience and neuromorphic engineering. Neuron, 109(4):571-57) 2021.
12

---

Published as a conference paper ICLR 2024
Hanle Zheng, Yujie Wu, Lei Deng. Yifan Hu, and Guoqi Li. Going deeper with directly-trained larger spiking neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 11062-11070, 2021.
Yaoyu Zhu, Zhaofei Yu, Wei Fang, Xiaodong Xie, Tiejun Huang, and Timothée Masquelier. Training spiking neural networks with event-driven backpropagation. In Advances Neural Information Processing Systems, 2022.
A THEORETICAL DERIVATION
To make it easy to follow, we write Assumption again in the following: Assumption A.1. Assume all entries of s' [t] (of size B BCC))) and W (of size On Cous) are independent for < < T, all [ obey i.i.d Bernoulli(p(t) distribution, and all w/; obey any i.i.d distribution.
For simplicity, omit the superscript 1 and - in the following derivation. Before proving the theorems, we derive the mean and variance of variable Isa in Lemma. A.2 and Lemma. A.3
Lemma A.2. When Assumption 4.7 holds, all Toe [t] will share the identical distribution, and [[]x[[]]]
(wyy VAR Aox [t]] Can (pt) (aRRWWAARWWu + (plt] p|t) "
Proof. Since [or[d] ビØ=ı $by [t] juys all sum of products of independent variables with identical distri- butions (sby(t) [t] and Wyz); they share the identical distribution. We calculate the mean and variance of [ss[t] as follows:
Cin
B [T,][[]] = Σ (sb, (()))))))))) = B(uyd
(17)
27
Cin
Cin
E[L E
sby tlys
E
Sby [t)))
+ Oin (Ctn 1)p(t(²B2(uu [1
Cin
Σ
[656 [( + Cin (Cin 1)
f=1
Cn + Can (Cin 1))lt| [w]j]]]]]]]]]]]
(18)
E[
[|]
E Th: (tt [|]] (w)d))
= =Cnn (plt) + (p(t) - p(()3))"(wo,,))
(19)
Lemma A.3. When Assumption 4. holds, for all 1 < b, by < B, 1 i2 S Cout. and 1 S t1, t2 < T,
I [t1] and Ibyia [ty] are uncorrelated when (b,, t1 + (b2,tz) and in F i2. When (bI, t1, (bz, t22
and in + i2, (OV( [4], Tnig (4) = Ci,, B. (B - r((]²); When (b,,1) + (b2, 12) and in in.
COV(In Thpi((a)) = Chapl1 ( (wrgl
Proof. Since Tue [1] Egn Sby Mgl4u1s..
Cen
COV(In Abgis Aoy (t2)
COV
Σ
Sbyj [t]] Wiin
Σ
sbzs [2a] Waa
(20)
13

---

Published as conference paper ICLR 2024
When (1) * t2) and ij * ig. the lemma is trivial since the entries in the summation are all uncorrelated. For the case when (b1, (1) = (b2, 12), we have:
)
Cin
Cin
COV [4] Jniy [1]) E
86y Maaee
Σ
sby sogl4
El [4] [4]
Cin
(sbb [t] Wiss, Wyrn)
(sby [t] (E(sby (E(sby(t))g)))
[pttE"/[ujj - p|t]/1"([uss))
=
C. E [10 [105 - -((13)
(21)
For the case when i = i2. we have:
Cin
Cin
COV(T,, Toge (t2)) = E
Σ
8b, (1a
Σ
Sbo (t2l
BT [E]] [T [t2]]
Cin
B [tsb)]th]sbbnle
E(sb., [t1 (B [tz (es]
Cin
Σ
p(tilp(ta)B((ss))
==1
(22)
After getting the mean, variance, and covariance of Isstt), we can prove the following theorems by calculating the coefficient before the variance of p(t]:
Theorem A.4.
When Assumption holds and the gross firing rate p holds constant, then the expectation of sample variance of pllt] among time-steps! E | ET= ult/ 2²] increases when the variance of firing rate among time-steps 1 ET=plt - xd increases.
Proof. Here omit the subscript b (the batch dimension) for [t] First calculate the expectation of u1[l] and p::
[[[ = B(Toa [t] = C... Cinp(t]EE(u [[]5]
(23)
1
[[p]
Σ
[[u[t]]
Ct.pEEuwj
(24)
T
14

---

Published as conference paper ICLR 2024
Then we calculate the second moment, including Blu[t]²] and Blulti [tal]:
Caus
Blu(d3] = R [n|d]]
VAR[u[]] = [[)]) +
VAR
T:(4]
Cous
Cane
=CnptttB'' [wggs] +
1
Σ
VAR (+(t) +2
E
Cout
Ish
1
Con (plt(EE [[t]((((w5))) + Cout (Cout 1)C (wsysa((tt - -plf())
Cout
Cin
- [wyy]] Cout
[[t]WAR[[sh) + Cour (plt) - p(
(25)
B|u(41][(1a]] [ + COV[M(4)) [12]]
Caut
Cous
Cov
Σ [1tt):
[t(t2]
C
Caut
tc +
COV [[tt2)
Co
Cin Cout
-Ch.itipp
pltspples/ARR(u
(26)
Finally, we can calculate the target function:
1
E
plt
22
E
plt
T
T
1=1
1=1
T
E[p[t]9
2
Σ
[[ult|/((te]
T
T
15h S
T-1
T
T
Cin Cout
(p1[VAR[ug)) + Cour (p(1) p((12)p² (R2(wss)))
+
T
2
Σ
Chellipp(()RBR [wsss]
Cin Cout
+
T2
T
T 7-1
-C2 B2 [un
plfj2 p² + CinWPlWual
Σ
pftj2
p
T
T
Cin WAl Cout
T-1 1 p
plf p²
(27)
When p is constant, the variance among time-steps only depends on Σ p[t]?_ the last equation of
Eq. 27 the only thing that can vary is ET= p[t] and the coefficient in front of it is always positive (C2 E [wyll= + 2 Ci,, E", + C., T Therefore, the conclusion holds.
15

---

Published as a conference paper ICLR 2024
Theorem A.5. When Assumption 4.1 holds and the gross firing rate p keeps constant, then the expectation of variance within time-steps 1 Ei= [2]]] keeps constant.
Proof. We first calculate Eio²"(t and then sum them up. The [[] can be split into calculating [[] and ]l which have been calculated before.
Cout
Cout
E|o² t]] = E
Σ i=1
410 - n/()?
[[n[t]]]
Cout
Cout
-Cinpit)B(uff + Can (Cin - 1))pt)
C (
Cin (pl)VAR((u)) + Cour =12
Cout
=Ompptt][uu Csm Cmplt Cin (pttVVAR[ujsl + Coue (1()))
Cout
Cin (p1]VAR(wg) (ujg) + Cous (pl (B2)))B2((3))))
Cin ( (x14) p(1]3 (B?((B()))))
Cout
Cin (Cout 1)
p p|]VAR[[ujd
(28)
Cout
T
1
Cin (Cont 1)
Cin Cin(Cout - 1) p VAR [uyd
[t]
(29)
Cout
Cout
As result, the variance of p(4] will not affect E E oi[1] which means it keeps constant.
B ALGORITHM DESCRIPTION FOR OUR METHOD
Our algorithm works under the online learning framework, which means the network goes through forward and backward propagations step by step from time-step 1 to T (instead of first forward from time step to T and then backward from time step T to 1). Since the network is processed step by step, it does not require saving the intermediate state from step to T as in regular BPTT. In each time-step, the information goes from the input layer to the output layer of the network in the forward pass, and then the gradients go from the output layer to the input layer in the backward pass.
The workflow of each layer is shown in Algorithm 1 while the calculation of p ft,02 0² is shown separately in Algorithm
C IMPLEMENTATION DETAILS
We conduct experiments on CIFARIO, CIFARION 100, DVS-Gesture, CIFARIODVS, and Imagenet datasets. The network structure of VGGSNN we use for the CIFARIO, CIFAR100, DVS-Gesture, CIFAR I0-ARO datasets is consistent with OTTT where 64C3 denotes convolution layer with 3 x convolution kernel and 64 output channels, AP2 means 2 2 average pooling, GAP means global average pooling, and FC means fully connected layer. For Imagenet classification, just use standard Resnet-34 architecture.
512CC-GAP-FCC
In all experiments, we use an SGD optimizer with a momentum of 0.9 with a cosine annealing learning rate scheduler. The data augmentation use for each dataset is listed follows: For CIFAR10 and
16

---

Published as conference paper ICLR 2024
Algorithm 1 The workflow of each layer
Input:
Output of the last layer s (input spike train/image time for the input layer) and the weight
[[]]
between last layer and current layer W (s'- and W are both tensors instead scalars).
// 1. Calculate input current [[]
[[]] layer(s' (4) W
layer means a conv layer, a linear layer or other types of layer
// 2. Apply OSR on [[t] to get the normalized [[]]
training then
Calculate ylt). [ fi, o² according to Algorithm.
r Vallt
714 -7 [14] no_grad Vo*lot+e
Mitl- ) -
nno_grad
Vaiae
Voete
else
7[t]
7 Voz+e
+ B
> Note that same linear transformations are applied in training and inference
end if
3. Update membrane potential of neurons in layer according to the LIF neuron model and input Z[t]
w (1 - wt 0.5] + [[t]
4. Apply OTS to update the threshold 0[t]
0[t] Jmeml [t] + mem -
Trom !|]
5/5. Fire spikes s'[t] and then reset membrane potential 8 (w'' [t] O((t)
u i 0.5] = u (1]((1 (()
Algorithm 2 The calculation of p(t), [tt,
Input: (Additional Input) Current time-step (1 TT
To determine whether we should initialize
variables or calculate running mean/variance at the current time-step
Output: (W( (n ......N)
1. Calculate the batch mean u(t] and variance of according to [t]] according to Eq. 13 (Here m is the
number of elements in channel, which forms group for normalization).
[4[t]
[t]
0 E
Ex=1 [Io[t]
2()))²
2. According to Eq. 15 we need variables and o² to accumulate total mean and variance. Before
the first time-step, we initialize y and d² to 0:
if t then
f1 0, +
end if
Then we accumulate total mean and variance according to Eq. 14
+- + +u(l), + o² + (a + p((]3)
3. Calculate running mean fi and running variance a² in time-step T (the last time step)
ift then
a² + o² 11
A - i
momentum) (u ji) - - d2 + (1 momentum)(o" - )²)
We take momentum = 0.9 in BN.
end if
17

---

Published as conference paper ICLR 2024
Table 3: Experimental configurations
Dataset
CIFARIO
CIFARI00
DVS-Gesture
CIFARIO-DVS
Imagenet
Epochs
300
300
300
300
100
Batch size
128
128
128
128
256
Learning rate
0,1
0.1
0.01
0.1
0.1
Weight decay
5e-4
5e-4
5e-4
5e-4
2c-5
MSE weight
0.05
0.05
0.001
0.001
0.05
Dropout rate
0
0.05
0.1
0
100, we use RandomCrop(4) + Cutout() + RandomHoorzzontllPiipp) + Normalize(); For DVS-Gesture, we use RandommesizeeCroppp ralDelete( (14) (recall the total time-step is 20, and the random temporal delete drops 6 time-steps (30% For CIFAR 10-DVS, For Imagenet, we use Kandommesizzedrrpp During testing, the image first resized to 256 x 256 and center-cropped to 224 224 and then normalized.
scale=(0.7. I.0)) + Resize(48) RandomRotatiion(0)) Random RandomTempo-
the neuromorphic data augmentation (NDA) which comes from (Li et al., 2022).
+ Randoom/orrizonaaFFiip + Normalize() during training.
Other hyperparameters we use are provided in Table 3 including total training epochs, batch size, learning rate, weight decay, (weight of MSE loss in Eq. 5) and dropout rate.
For the configuration of Vanilla BN used as the baseline, calculates the statistics solely based on data from each time step. In this approach, at every time step during training, the normalized input [[] is computed by
[|t= - [
plt) +
+ 3,
where the batch mean su[t] and batch variance [t at time-step t accords with Algorithm 2 in the above.
The slight difference between it and the standard BN the calculation of rumning mean and variance: we use the same momentum as in our OSR to update running mean and variance at each time step, they will be unstable since they are updated T times more compared with our OSR. A simple approach is to change the momentum parameter to (1 momentum)/" but we choose to implement it in strict corresponding way: We accumulate the mean and variance during training (for the variance, we accumulate by o² f- o² + [ instead of the complex way shown above), and only update the running mean and
variance at time step T. In this way, there no need to change the momentum parameter.
D ABLATION STUDY
Here we show the ablation results of our proposed modules: online spiking renormalization (OSR) and online threshold stabilizer (OTS). Since the OTS is proposed to help the training of OSR, we provide the results of Vanilla BN / OSR / OSR+OTS here on CIFAR10/100 and Imagenet dataset. The results are shown in Table.
4 It is shown that adding OSR will improve the performance over vanilla BN, and adding OTS will further improve the performance over solely adding OSR. It worth noting that only adding OSR is sensitive the weight decay parameter, it often requires lower weight decay to get a better result. For example, both Resneet199+ 19+(V9illll Vanilla BN) and Resnet- 19+OSR+OTS can be trained on CIFAR 10 with a weight decay of cannot. Another example is that the performance of NEW-Resnne-334-OS.
2e-4 while Resnett-1+OOS degrade to 54.97% on Imagenet when using weight decay of 2e-5. On the other hand, OSRS much more stable with large weight decay parameters.
will
18

---

Published as conference paper ICLR 2024
Table 4: Ablation results
CIFARIO Ace (wd)*
CIFARI00 Acc (wd)
Imagenet Acc (wd)
VGG+Vanilla BN
92.6 (5e-4)
75.17 (5e-4)
VGG+OSR
94.05 (2e-4)
75.65 (2e-4)
VGG+OSR+OTS
94.35 (5e-4)
76.48 (5e-4)
cosne-19+Vanille BN
92.96 (2e-5)
73.68 (2e-4)
ReSR
95.14 (2e-5)
74.03 (2e-5)
Resnet- O
95.20 (2e-5)
77.86 (2e-4)
SEW-Resnnet-3+-anill BN
60.48 (2e-5)
SEW-Rcsnet:34+OSR
61.92 (0)
SEW NEW--Resnet-44+OSR-OTT
64.14 (2e-5)
report both accuracy and weight decay statisties here.
ADDITIONAL EXPERIMENTS
The OSR mechanism is shown to be useful among many mechanisms that have tried. One simpler mechanism that does not work well is directly using "linear transformation" instead the "double transformation" " in OSR. directly applies
Necessity of double transformation" OSR.
1 i
- -
V + +
in both training and inference. We have tested this approach on the CIFARI00 dataset using VGGSNN and find it very hard to train. The final result we get is 53.25%, which significantly worse than OSR.
Fixed f[t] during inference in OTS. In OTS, the threshold 0[t] is dynamically adjusted for each sample batch during both the training and inference phases. will better when {[t) fixed during the inference stage if there is no significant performance decrease since inference batch size will not affect performance and it is more friendly to neuromorphic chips under this case. Hence we have conducted two extra experiments:
1. We have tested the performance of using fixed running 0[t] on Imagenet (using our saved model), its performance is 64.06% (original accuracy is 64%) (using the saved model of OSR+OTS). This result shows that fixed f works well.
2. We have tested the performance for batchsize on Imagenet (also using saved model), and the performance is 62.64%. Although there is a performance drop, it is still better than the baseline.
F MEMBRANE POTENTIAL VISUALIZATION
To see whether the Gaussian assumption in OTS is reasonable, we collect the membrane potential of a VGG network trained with OSR and OTS on the CIFAR-10 10 dataset and visualize the distribution of membrane potentials for each layer and each time step. The results are shown in Fig. These distributions display the shape of bell curves, which indicate the similarity between these distributions and Gaussian distributions. Most of the distributions take the mean value around zero. This result shows that the Gaussian assumption in OTS is reasonable. Therefore, our proposed algorithm exploits the adaptation of batch normalization and can cope with the varied distributions of network features during online learning of spiking neural networks.
19

---

Published as a conference paper ICLR 2024
T=0
0.8
T=0
T=2
1-2
0.4
0.8
Daraty
Darety
0.3
0.4
0.2
0.2
0.
(a) Layer 1
(b) Layer 2
0.7
T=0
T40
0.8
T-2
T-2
0.4
Damily
0.4
0.3
0.2
0.1
0.0
0.0
(c) Layer 3
(d) Layer 4
T-0
T-0
0.8
T=2
T-2
Damiiy
Dimnity
0.8
0.8
0.4
0.2
0.2
0.0
0.0
(c) Layer 5
Layer 6
1.75-
T-0
1.75
T-0
1.50
T=1
T-1
T-2
1.50
T-2
1.25
Danaly
1,00
T+3
Denaly
1.25
T+3
1.00
0.75
0.75
0.50
0.5
0.25
0.00
-2
2
0.00
-2
2
(g) Layer
(h) Layer 8
Figure 3: Visualization of the distributions of membrane potentials for each layer and each time step.
20


Reviews:
[{"review_id": "7upFxRmgur", "reviewer": "Reviewer_9e6u", "summary": "The work focuses on online training techniques, especially addressing the limitation of not having access to future information in early time steps in online training. This paper tries to incorporate BN into online training, by two new modules, Online Spiking Renormalization (OSR) and Online Threshold Stabilizer (OTS).  \nThis online training setting will benefit memory consumption. However, there are several problems: \nthe presentation of the new idea is not clear at all, it depends on the running mean and running variance, but how to calculate them is not mentioned clearly. we still don't know how to save memory by using the proposed modules. \nThe other issue is that in the theoretical parts, authors tend to make unrealistic assumptions. How can the spike train be iid. and expect the weights to be iid. Another thing is that the conclusions of the theorems, why do we need expectation of the average of u[t], and expectation of the average of sigma[t], how can these results help with the main new modules?\nFor the experiments, there is only one method using the same structure as the new method, shouldn't compared with different network architectures? Shouldn't compare with different BN methods? That is absent in the main paper.", "questions": "This online training setting will benefit memory consumption. However, there are several problems: \nthe presentation of the new idea is not clear at all, it depends on the running mean and running variance, but how to calculate them is not mentioned clearly. we still don't know how to save memory by using the proposed modules. \nThe other issue is that in the theoretical parts, authors tend to make unrealistic assumptions. How can the spike train be iid. and expect the weights to be iid. Another thing is that the conclusions of the theorems, why do we need expectation of the average of $u[t]$, and expectation of the average of $\\sigma[t]$, how can these results help with the main proposed modules?\nFor the experiments, there is only one method using the same structure as the new method, shouldn't compared with different network architectures? Shouldn't compare with different BN methods? That is absent in the main paper. \nHow to calculate $\\hat{u}, \\hat{\\sigma}$? please explain. \nIs $\\gamma$, $\\beta$ learnable or fixed?\nNo point of Online Threshold Stabilizer (OTS) to stabilize the firing rate of each layer?\nNo experiment results on resnet on cifar datasets.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "This paper proposes a novel approach to training spiking neural networks that is memory-efficient and biologically plausible. The proposed Online Spiking Renormalization and Online Threshold Stabilizer techniques ensure consistent parameters and stable neuron firing rates across time steps.", "weaknesses": "The main idea should be better explained to make it easier to understand. \nHow to compute gradients in the backward stage if we use this forward transformation? The authors asked themselves the question, but did not give clear answers. Please make the core part of the paper clear.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FPNWATNoip", "reviewer": "Reviewer_2ic1", "summary": "The paper considers the online setting of SNN, and points out one important mismatch of BN happens in the training and testing stages. The authors proposed one nice solution to solve the issue, and the experimental results support the benefits with the new algorithms. Importantly, in addition to the experimental verification, the authors also provide necessary theoretical analysis to the new algorithm.", "questions": "Please find the comments above", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. The paper have a very clear motivation: mismatch of BN happens in the training and testing stages. \n2. The authors proposed one nice solution to solve the issue, and the experimental results support the benefits with the new algorithms. \n3. The propblem of this paper is unsolved in the comminity before. The solution of this paper is interesting and novel. Importantly, it signicantly improve the performance. Considering the important role of BN in ML, I think the signicance and novelty of this work is good.\n4. The authors also provide the necessary theoretical analysis to the new algorithm.\n5. Presentation is good. Especially, I like figure 1 to intutiively explain the proposed algorithm.", "weaknesses": "The paper is with a nice story, and I especially like the figure 1 to intutiively explain the proposed algorithm. However, I still have some questions or comments below.\n\n1. What is the intuitive reason to have double normalization in (12)\n2. I suggest to provide the experimental results to verify the Gaussion assumption.\n3. I did not find the calculation form of $\\hat{\\mu}$ and $\\\\hat{\\sigma}$. I suggest to provide a clear definition of them in the paper.\n\nMinor issue:\n1. I did not find  the meaning of m in (8).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GUOmPOW5EK", "reviewer": "Reviewer_pFYW", "summary": "This paper focuses on the online training of SNNs. It adds the normalization mechanism into online training, which have not yet been fully explored by previous works. It proposes two modules to improve the standard Batch Normalization, named Online Spiking Renormalization and Online Threshold Stabilizer, which ensure consistent parameters and stable neuron firing rates across time steps. The paper demonstrates the effectiveness of the proposed methods on various datasets and shows that they outperform existing state-of-the-art online training methods.", "questions": "1. What is the difference between OSR and batch renormalization [1] ?\n\n2. The online calculation of all-time mean and variance is interesting. However, where is it used? Is it a part of OSR?\n\n3. The authors have made an assumption that the membrane potential follows a normal distribution in OTS. Although experiments have shown the effectiveness of OTS+OSR, I am still curious about the real distribution of the membrane potential.\n\n[1]Sergey Ioffe. Batch renormalization: Towards reducing minibatch dependence in batch-normalized models. Advances in neural information processing systems, 30, 2017\n\nI consider increasing my score if the authors can solve my concerns.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper presents a novel approach to training spiking neural networks that integrates essential batch normalization into the online training process by introducing online spiking renormalization and online threshold stabilizers to enhance training stability.\n\n2. The paper is well-organized and well-written. Also, the figures and tables are well-designed and provide a clear representation of the results.\n\n3. The proposed method outperforms existing state-of-the-art online training methods.", "weaknesses": "1. Although the online training approaches save the memory cost, the proposed method falls short of BPTT in performance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2X51jPHaiN", "reviewer": "Reviewer_W97A", "summary": "When batch normalization (BN) is implemented in spiking neural network (SNN) structures, it is a common practice to compute the statistics across all time steps for running BN. However, this commonly used BN is ill-suited for the online training of SNNs. To enable BN for online training, this paper introduces two strategies called Online Spiking Renormalization (OSR) and Online Threshold Stabilizer (OTS), which preserve the online training property and memory efficiency. Experimental results demonstrate the efficacy of the proposed methods.", "questions": "1. The line right before Section 4.3: is there typos in the formula? Should the formula be $...(\\theta[1]-\\mu[1])/\\sigma[1]$? Then does it mean that $\\theta[1]$ is fixed all the time?\n2. How to implement the proposed method on NF-Resnet-34 which is a normalizer-free architecture? The reviewer knows that the goal of this experiment is to compare the proposed method and OTTT. But still, the proposed method and normalizer-free nets are totally orthogonal.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The proposed tricks are both simple to implement and logically sound, contributing to their practicality and ease of use.\n2. The performance is good when compared to online training methods and conventional methods.", "weaknesses": "1. If the reviewer understands the OTS method correctly, the threshold $\\theta[t]$ is dynamically adjusted for each sample batch during both the training and inference phases. However, the reviewer thinks that $\\theta[t]$ should be precalculated and fixed during the inference stage. Otherwise, the batch size will highly influence the performance. Much worsely, the obtained network cannot be implemented on normal neuromorphic chips because the chips do not support calculating the mean and variance. the reviewer suggests the authors to maintain a running $\\theta[t]$ used for inference.\n\n\n2. This paper could be regarded as an engineering work. Then the reviewer thinks that the ablation experiments are not abundant:\n (i). When OSR is incorporated individually, there appears to be a significant performance degradation. It is important to clarify whether such a phenomenon is typical across various datasets.\n (ii). A simpler approach to integrating BN into the online training regime involves calculating the statistics solely based on data from each time step. In this approach, at every time step during training, the normalized I[t] is computed based on $\\mu[t]$ and $\\sigma[t]$, as illustrated in eq. 8. Additionally, the running mean and variance are also updated at each time step. This vanilla BN method should be considered as the baseline. It would be valuable to assess whether this method outperforms OSR and whether the combination of the baseline with OTS outperforms the combination of OSR with OTS.\n (iii). In the backward stage, the authors utilize a \"double transformation\" trick to facilitate more meaningful backpropagation. What if we directly conduct bachprop based on the ``linear transformation''? Is OSR better than that?\n\n\nMinor:\n1. eq.2: $s^{l-1}[t]W^l$ -> $W^ls^{l-1}[t]$, eq.4: $u^{l} [t] (1-s^{l} [t] )$  -> $u^{l}[t] \\odot (1-s^{l}[t])$\n2. The ``Online Calculation of All-time Mean and Variance'' part appears somewhat trivial. The reviewer thinks that it might be unnecessary to include this information in the list of contributions. The audiences may expect a more significant contribution when such a detail is highlighted.\n3. The statement preceding Section 5: ``our OTS mechanism helps our OTS mechanism''.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (spotlight)