SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Here 'label' is the index of the labeled neuron, and  $t_{k_0}(s_i)$  is a representative spike timing since all  $\frac{\partial L}{\partial t_k(s_i)}$  are equal. The detailed proof is provided in the Appendix. This gradient sum on label output neurons is a quadratic function of the number of output spikes  $m$ , which provides too much positive gradient on spike timing when  $m$  is large.
Positive sum of gradients on spike timing. When the network is nearly converged, output neurons corresponding to the labels often emit much more spikes than other neurons, which implies that the slope of membrane potential  $\frac{du}{dt}$  for these neurons at spike time is usually more significant than others. As a result, the  $\frac{\partial t}{\partial u}$  part on the gradient propagation path (refer to Figure 1) for label neurons is smaller than others in scale (but their signs are both negative). If the sum of gradients is exactly zero, the weights in the last layer will receive more positive gradients than negative ones, leading to decreasing weights in the last layer. This effect will inhibit output neurons from emitting spikes.
4.4 Enhanced Counting Loss
Based on the analysis in Section 4.3, we infer that loss functions providing adequate positive overall gradients are more suitable for time-based training schemes. Building upon this conclusion, we introduce a novel loss function to optimize network training. Since the counting loss (used in [69]) provides too much positive gradient on spike timing when the label output neuron emits many spikes, it is more adequate to let  $\sum_{s_i(t_k)}\frac{\partial L}{\partial t_k(s_i)}$  be a linear function  $2\lambda (\mathrm{target}_i - m)$  instead of a quadratic function  $2\lambda m(\mathrm{target}_i - m)$  in Eq. (12). To do this, suppose a neuron fires  $m$  spikes, we can substitute  $\frac{\partial L(m)}{\partial m} = 2(m - \mathrm{target}_i)$  with  $\frac{\partial L(m)}{\partial m} = \frac{2(m - \mathrm{target}_i)}{m}$ . The form of this modified loss function is
$$
L (\boldsymbol {s}, \mathbf {t a r g e t}) = 2 \lambda \sum_ {i = 1} ^ {C} f \left(\int_ {0} ^ {T} s _ {i} (t) d t, \operatorname {t a r g e t} _ {i}\right). \tag {13}
$$
$$
\text {w h e r e} f (x, \text {t a r g e t}) = \left\{ \begin{array}{l l} x - \text {t a r g e t} \ln (x), & x > 0 \\ 0, & x \leq 0 \end{array} \right. \tag {14}
$$
with derivative  $\frac{\partial\int s_i(t_k)}{\partial t_k(s_i)} = -1$ . We name the loss as enhanced counting loss. The detailed equation-deducing procedure is provided in the Appendix.
4.5 Moving Scaling on Weights to Thresholds
In previous works, weight standardization [45] is used [69] to stabilize training since common normalization schemes [24] are not applicable in asynchronous networks. The weights of a layer are decided by three sets of parameters  $\tilde{W},\gamma ,\beta$  , which correspond to the original weight, scale, and shift:
$$
\hat {W} ^ {(k)} = \frac {\tilde {W} ^ {(k)} - E [ \tilde {W} ^ {(k)} ]}{\sqrt {\operatorname {V a r} [ \tilde {W} ^ {(k)} ] + \varepsilon}}, \quad W ^ {(k)} = \gamma^ {(k)} \cdot \hat {W} ^ {(k)} + \beta^ {(k)}. \tag {15}
$$
Here  $x^{(k)}$  means the  $k$ -th channel of  $x$ ,  $\hat{W}^{(k)}$  is the normalized weights for  $\tilde{W}^{(k)}$ , and the final weight  $W^{(k)}$  goes through a linear transform with the scale  $\gamma^{(k)}$  and shift  $\beta^{(k)}$ . In time-based training (Figure 1(c)(d)), weights participate in backpropagation by passing gradients from output spike timing to membrane potential and then to input spike timing of a neuron. According to Eqs. (2) (5), we have:
$$
\frac {\partial t _ {k} \left(s _ {i} ^ {(l)}\right)}{\partial u _ {i} ^ {(l)} \left(t _ {k}\right)} = \frac {- 1}{d u _ {i} ^ {(l)} \left(t _ {k}\right) / d t} = \left(\sum_ {j, t _ {k, \text {l a s t}} \left(s _ {i} ^ {(l)}\right) <   t _ {m} \left(s _ {j} ^ {(l - 1)}\right) \leq t _ {k} \left(s _ {i} ^ {(l)}\right)} \frac {\partial u _ {i} ^ {(l)} \left(t _ {k}\right)}{\partial t _ {m} \left(s _ {j} ^ {(l - 1)}\right)}\right) ^ {- 1}, \tag {16}
$$
$$
\frac {\partial u _ {i} ^ {(l)} \left(t _ {k}\right)}{\partial t _ {m} \left(s _ {j} ^ {(l - 1)}\right)} = w _ {i j} ^ {(l)} \cdot \frac {\partial \epsilon \left(t _ {k} - t _ {m}\right)}{\partial t _ {m}}. \tag {17}
$$
Therefore the change in the weight scale will influence the gradient backpropagation by changing  $w_{ij}^{(l)}$  in Eq.17. However, threshold values do not appear in the above backpropagation paths. Thus, learning thresholds instead of weights might stabilize the training process. In forward formulas Eqs. (1) (2), scaling up weights  $w_{ij}^{(n)}$  and thresholds  $\theta$  by the same factor will not change the output spike train. The weight scale is determined by  $\gamma$  and  $\beta$ , so we can transform to keep  $\gamma$  constant and learn  $\theta$ :
$$
\theta^ {(k)} \leftarrow \frac {\gamma 0}{\gamma^ {(k)}} \cdot \theta^ {(k)}, \beta^ {(k)} \leftarrow \frac {\gamma 0}{\gamma^ {(k)}} \cdot \beta^ {(k)}, \gamma^ {(k)} \leftarrow \gamma_ {0}, \tag {18}
$$
where  $\gamma_0$  is the value we want to fix  $\gamma$  to. When doing parameter updates, denote  $d\gamma^{(k)} = \frac{\partial L}{\partial\gamma^{(k)}}$  and  $d\beta^{(k)} = \frac{\partial L}{\partial\beta^{(k)}}$ , the parameter update rule after the above transformation can be described by:
$$
d \theta_ {\text {n e w}} ^ {(k)} = d \left(\frac {\gamma_ {0}}{\gamma^ {(k)}} \theta^ {(k)}\right) = \theta^ {(k)} \frac {- \gamma_ {0}}{(\gamma^ {(k)}) ^ {2}} d \gamma^ {(k)} = - \frac {\theta^ {(k)}}{\gamma^ {(k)}} d \gamma^ {(k)}, \tag {19}
$$
$$
d \beta_ {\text {n e w}} ^ {(k)} = d \left(\frac {\gamma_ {0}}{\gamma^ {(k)}} \beta^ {(k)}\right) = \frac {\gamma_ {0}}{\gamma^ {(k)}} d \beta^ {(k)} + \beta^ {(k)} \frac {- \gamma_ {0}}{(\gamma^ {(k)}) ^ {2}} d \gamma^ {(k)} = d \beta^ {(k)} - \frac {\beta^ {(k)}}{\gamma^ {(k)}} d \gamma^ {(k)}. \tag {20}
$$
It should be noticed that the value of  $\gamma^{(k)}$  will be unchanged (always equal to  $\gamma_0$ ) during learning after this transformation. During learning, parameters to be learned are  $\theta$  and  $\beta$  where  $\theta^{(k)} \gets \theta^{(k)} + \eta d\theta_{\mathrm{new}}^{(k)}$  and  $\beta^{(k)} \gets \beta^{(k)} + \eta d\beta_{\mathrm{new}}^{(k)}$ .
4.6 Overall Algorithm
Algorithm 1 Discrete Version of Our Algorithm
Input:  $s^{(0)}$  : Input spike train,  $W^{(n)}(n = 1,\dots,N)$  : Weight for all layers (where  $N$  is the number of layers), Total simulation time  $T$    
Output:  $\nabla W^{(n)}(n = 1,\dots,N)$    
// Forward propagation:   
Initialize Neuron(n)(n = 1,...,N): Neurons with states (membrane potential parameters) for all layers   
for  $n = 0$  to  $N - 1$  do for  $t = 0$  to  $T - 1$  do Update state of Neuron(n+1) according to Eq. 1 if Membrane potential  $\pmb{u}^{(n + 1)}[t]\geq \theta$  then >Fire spikes  $\pmb{s}^{(n + 1)}[t] = 1$  Reset state of Neuron(n+1) end if end for   
end for   
// Backward propagation:   
Initialize  $\nabla W^{(n)}\gets 0,\frac{\partial L}{\partial t(s)^n}\gets 0$  for  $n = 1,2,\ldots ,N$    
Calculate  $\frac{\partial L}{\partial t(s)^N}$  according to Eq. 13 14   
for  $n = N$  to 1 do for  $t = T - 1$  to 0 do if  $s^{(n)}(t) = 1$  then Spikes fired Set  $\frac{\partial L}{\partial u^{(n)}} [t]$  to  $\frac{\partial L}{\partial t(s^{(n)})}\frac{\partial t(s^{(n)})}{\partial u^{(n)}(t)}$ $\begin{array}{r}\triangleright \frac{\partial L}{\partial t(s^{(n)})}\frac{\partial t(s^{(n)})}{\partial u^{(n)}(t)} \end{array}$  is part of Eq. 3 4 else Set  $\Delta t\gets \Delta t + 1$  end if //  $\frac{\partial u^{(n)}}{\partial W^{(n)}}$  and  $\frac{\partial u^{(n)}}{\partial t(s^{(n - 1)}}$  both involve  $\epsilon (\Delta t)$  where  $\epsilon (\cdot)$  is the spike response kernel Set  $\nabla W^{(n)}\gets \nabla W^{(n)} + \frac{\partial L}{\partial u^{(n)}}\cdot \frac{\partial u^{(n)}}{\partial W^{(n)}}$  >Eq.3 Set  $\frac{\partial L}{\partial t(s^{(n - 1)})}\gets \frac{\partial L}{\partial u^{(n)}}\cdot \frac{\partial u^{(n)}}{\partial t(s^{(n - 1)})}$  >Eq.4   
end for   
Update  $\theta ,\beta$  according to Eq. 19 20 as well as  $\tilde{W}$  using  $\nabla W^{(n)}$    
end for
Algorithm 1 describes our overall algorithm. Our algorithm works in discrete time-steps here since it suits better for current deep learning frameworks such as PyTorch. The main training and inference workflow follows [69]. The enhanced counting loss is in the beginning of the backward propagation and the transfer of training from scaling factor to threshold is embodied in the second last line.
We conduct experiments on MNIST [31], Fashion-MNIST [62], NMNIST [41], CIFAR10/100 [30], DVS-Gesture [1], and CIFAR10-DVS [32] datasets to evaluate the performance of our method. The methods we compare are Spiking Neural Networks trained with time-based gradients carried by spikes, including TC [39], ASF [8], S4NN [27], BS4NN [28], STiDi-BP [38], STDBP [66], TSSL-BP[67], and EDBP [69]. All experiments are run on single Nvidia RTX 3080Ti/3090 GPUs. The implementation details are provided in the Appendix.
5.1 Compare with the State-of-the-Art
We first compare our method with current state-of-the-art time-based SNN training approaches. Table. 1 reports the results on MNIST, Fashion-MNST, N-MNIST, CIFAR10, CIFAR100, DVS-
Table 1: Performance comparison on MNIST and CIFAR-10/100 datasets
Dataset	Model	Gradient Type	Architecture	Accuracy
MNIST	TC [39]	Temporal	784-800	97.5%
ASF [8]	Temporal	784-340	97.9%
S4NN [27]	Temporal	784-600	97.4%
BS4NN [28]	Temporal	784-600	97.0%
STiDi-BP [38]	Temporal	784-500	97.4%
STDBP [66]	Temporal	16C5-P2-32C5-P2-800-128a	99.4%
TSSL-BP [67]	Temporal	15C5-P2-40C5-P2-300	99.53%
EDBP [69]	Temporal	15C5-P2-40C5-P2-300	99.47%
ANN	-	15C5-P2-40C5-P2-300	99.50%
Ours	Temporal	15C5-P2-40C5-P2-300	99.50%
Fashion-MNIST	S4NN [27]	Temporal	784-1000	88.0%
BS4NN [28]	Temporal	784-1000	87.3%
STDBP [66]	Temporal	16C5-P2-32C5-P2-800-128	90.1%
TSSL-BP [67]	Temporal	32C5-P2-64C5-P2-1024	92.83%
EDBP [69]	Temporal	32C5-P2-64C5-P2-1024	93.28%
ANN	-	32C5-P2-64C5-P2-1024	93.94%
Ours	Temporal	32C5-P2-64C5-P2-1024	94.03%
N-MNIST	TSSL-BP [67]	Temporal	12C5-P2-64C5-P2	99.40%
EDBP [69]	Temporal	12C5-P2-64C5-P2	99.39%
Ours	Temporal	12C5-P2-64C5-P2	99.39%
CIFAR10	TSSL-BP [67]	Temporal	CIFARNetb	91.41%
EDBP [69]	Temporal	VGG11b	92.10%
ANN	-	VGG11	94.51%
Ours	Temporal	VGG11	93.54%
CIFAR100	EDBP [69]	Temporal	VGG11	63.97%
ANN	-	VGG11	74.23%
Ours	Temporal	VGG11	70.50%
DVS-Gesture	Ours	Temporal	VGG11	97.22%
CIFAR10-DVS	Ours	Temporal	VGG11	76.30%
a 15C5: convolution layer with 15 channels of  $5 \times 5$  filters; P2: pooling layer with  $2 \times 2$  filters
b The detailed network architectures are provided in the Appendix (section Experimental Settings)
Gesture, and CIFAR10-DVS datasets. It should be noticed that we do not list BPTT-based methods here since they make use of information when neurons do not fire spikes in the gradient propagation, which certainly benefits the training process and the final performance. On the easy MNIST and N-MNIST datasets, the performance of our algorithm is comparable to those of TSSL-BP [67] and EDBP [69]. The improvement is not noticeable as the performance is saturated ( $>99\%$ ). On other datasets, our algorithm evidently outperforms other time-based SNN training algorithms. For the Fashion-MNIST dataset, our algorithm outperforms the previous time-based SOTA algorithm [69] by  $0.75\%$ . For the CIFAR10 dataset, we achieve  $93.54\%$  top-1 accuracy with a VGG11 network, outperforming EDBP [69] by  $1.44\%$  with the same network architecture. Besides, we have achieved  $70.50\%$  accuracy on the CIFAR100 dataset, outperforming the previous SOTA SNN trained with a time-based scheme by more than  $6\%$  with the same architecture. In addition, we are the first time to train SNNs with time-based schemes on both the DVS-Gesture and CIFAR10-DVS datasets, achieving  $97.22\%$  accuracy on the DVS-Gesture dataset and  $76.30\%$  accuracy on the CIFAR10-DVS dataset. All these results demonstrate that the proposed method outperforms the state-of-the-art accuracy on nearly all datasets.
In this section, we study the effects of our proposed loss function and normalization scheme on the network performance. We select EDBP [69] as the baseline in our ablation experiments, which uses counting loss and weight normalization. The performance of SNNs with different loss functions and normalization schemes is illustrated in Table. 2 and Figure 3(a). It can be seen that the proposed enhanced counting loss outperforms the original counting loss on CIFAR100 to a large extent. We achieve  $68.95\%$  top-1 accuracy, which is higher than the baseline by  $4.97\%$ . At the same time, our proposed threshold normalization scheme is also effective in increasing the network performance.
(b)
Figure 3: Ablation results. (a) Test accuracy along with training epoch index. (b) Accumulated sum of absolute gradient values along with training epoch index. (c) Accumulated sum of gradient values along with training epoch index. (d) The ratio of positive gradients along with training accuracy. In the figures, the blue line indicates the counting loss and threshold normalization scheme; the green line indicates the enhanced counting loss and weight normalization scheme; the orange line indicates the enhanced counting loss and threshold normalization scheme.
To further illustrate how our proposed methods help the network training, we analyze the gradient scale as well as the ratio of positive gradients. Figure 3(b) plots the gradient scale described by the  $L1$ -norm of gradients, which is the sum of the absolute gradient value. Also, we provide the sum of (not absolute) gradient values in Figure 3(c). Both values are plotted against the
Table 2: Ablation study on the CIFAR-100 dataset
Loss Function	Normalization	Accuracy
Count (MSE)	Weight	63.98%
Count (MSE)	Threshold	66.35%
Enhanced	Weight	68.95%
Enhanced	Threshold	70.50%
number of training epochs, and they are both accumulated through epochs. From the figures, we can find that the enhanced counting loss greatly reduces the overall scale of gradients. Besides, the proposed threshold normalization scheme also slightly reduces the overall gradient scale.
We also show the ratio of positive gradients among all gradients in Figure. 3(d). It should be noticed that we do not plot it against the epoch index, instead, we plot it against the network training accuracy. The underlying reason is that this ratio is largely influenced by the ratio between the number of spikes that label output neurons emitted and other output neurons emitted. The spike emission status comparison between label neurons and other neurons in the output layer can be better described by the network training accuracy compared with the epoch index, especially when the convergence rates in different settings are different. Besides, we do not accumulate gradient data in this figure since we want to get the information in each epoch. From this figure, we can see that in most cases, gradients trained with the enhanced counting loss and the threshold normalization scheme are more balanced (closer to 0.5 in the figure). There is a rising tendency for the proportion of positive gradients with the convergence of the network, which is influenced by more spikes emitted by label output neurons. In the very beginning, the negative part of gradients is much larger than the positive part. Oppositely, the overall gradients mainly consist of the positive part near convergence.
In this paper, we demonstrate the applicability of rate-coding-based loss functions to time-based training. We analyze loss function properties, introducing the enhanced counting loss to replace mean square counting loss. Furthermore, we transfer scale factor training in weight standardization to thresholds. Experiments show significant network performance improvements on datasets like CIFAR100, achieving state-of-the-art results on most others. While we've made notable progress, there's potential for further research into the optimal positive gradient ratio during training and the effectiveness of loss functions for SNN training in the time-based scheme. Additionally, extending our approach to ANNs, as demonstrated in [25], holds promise.
[66] Malu Zhang, Jiadong Wang, Jibin Wu, Ammar Belatreche, Burin Amornpaisannon, Zhixuan Zhang, Venkata Pavan Kumar Miriyala, Hong Qu, Yansong Chua, Trevor E. Carlson, and Haizhou Li. Rectified linear postsynaptic potential function for backpropagation in deep spiking neural networks. IEEE Transactions on Neural Networks and Learning Systems, pages 1-12, 2021.
[67] Wenrui Zhang and Peng Li. Temporal spike sequence learning via backpropagation for deep spiking neural networks. Advances in Neural Information Processing Systems, pages 12022-12033, 2020.
[68] Shibo Zhou, Xiaohua Li, Ying Chen, Sanjeev T Chandrasekaran, and Arindam Sanyal. Temporal-coded deep spiking neural network with easy training and robust performance. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 11143-11151, 2021.
[69] Yaoyu Zhu, Zhaofei Yu, Wei Fang, Xiaodong Xie, Tiejun Huang, and Timothee Masquelier. Training spiking neural networks with event-driven backpropagation. Advances in Neural Information Processing Systems, 2022.