SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Yaoyu Zhu
Beijing, China 100871
yaoyu.zhu@pku.edu.cn
Wei Fang
fwei@pku.edu.cn
Tiejun Huang
tjhuang@pku.edu.cn
Xiaodong Xie
donxie@pku.edu.cn
Zhaofei Yu *
Institute for Artificial Intelligence
yuzf12@pku.edu.cn
Spiking Neural Networks (SNNs) are considered promising brain-inspired energy-efficient models due to their event-driven computing paradigm. The spatiotemporal spike patterns used to convey information in SNNs consist of both rate coding and temporal coding, where the temporal coding is crucial to biological-plausible learning rules such as spike-timing-dependent-plasticity. The time-based training strategy is proposed to better utilize the temporal information in SNNs and learn in an asynchronous fashion. However, some recent works train SNNs by the time-based scheme with rate-coding-dominated loss functions. In this paper, we first map rate-based loss functions to time-based counterparts and explain why they are also applicable to the time-based training scheme. After that, we infer that loss functions providing adequate positive overall gradients help training by theoretical analysis. Based on this, we propose the enhanced counting loss to replace the commonly used mean square counting loss. In addition, we transfer the training of scale factor in weight standardization into thresholds. Experiments show that our approach outperforms previous time-based training methods in most datasets. Our work provides insights for training SNNs with time-based schemes and offers a fresh perspective on the correlation between rate coding and temporal coding. Our code is available at https://github.com/zhuyaoyu/SNN-temporal-training-losses.
Artificial Neural Networks (ANNs) have achieved remarkable progress in various fields, such as computer vision [23], natural language processing [58], and reinforcement learning [51]. Compared with the conventional non-spiking ANNs, Spiking Neural Networks (SNNs) exhibit a greater level of biological plausibility [65] and achieve significantly improved energy efficiency on neuromorphic chips [9, 17, 37, 44, 49]. The benefit of SNNs on computation cost and power consumption originates from their event-driven nature to a large extent [47].
ANNs use real values to encode the state of a neuron. This encoding scheme is similar to the firing rate of a biological neuron. Inspired by this encoding scheme, researchers have applied ANN-SNN
conversion to obtain the parameters of SNNs [7, 22, 52]. Unlike firing rate coding, temporal coding is a unique feature of SNNs (compared with ANNs), which utilizes the temporal information of spikes in SNNs and encodes information by the absolute and relative firing times between spikes. The most commonly used temporal coding in SNNs is time-to-first-spike (TTFS) coding [3, 39, 67], which encodes the information by the firing time of the first spike emitted by a neuron. Although the encoding schemes in SNNs are richer than ANNs, directly training SNNs is harder than ANNs due to the binary nature of spikes and the non-differentiable membrane potential at spike time. Therefore, SNNs has not reached the performance of their ANN counterparts in classification tasks [10, 63].
There are two major categories of supervised learning techniques to train SNNs: The first category is activation-based learning, which treats SNNs as binary-output recurrent neural networks (RNNs) and uses backpropagation-through-time (BPTT) to train SNNs [2, 16, 19, 59]. The main difference between training SNNs and RNNs with these methods is that they apply continuous surrogate derivatives to substitute the discontinuous spike firing process [40]. Although competitive performances are achieved on the CIFAR-10/100, and even the ImageNet dataset [11, 14], these methods do not follow the asynchronous nature of SNNs. Firing-rate-based coding is intensively used in these approaches. In the output stage, the commonly used loss functions include spike-train-based ones [50, 64], spike-count-based ones [59, 60], and those defined on discrete time steps [11, 15]. All these loss functions behave in a rate coding scheme and have no direct relation to spike firing time (which indicates temporal coding). The second category is spike-time-based learning, which considers spikes' timing as an information carrier for parameter update in SNNs [3, 8, 61]. They are more biologically plausible since they keep the event-driven nature of biological learning rules [69]. Originated in SpikeProp [3], researchers try to solve the discontinuity at spike firing time and gradient vanishing/explosion in backpropagation. In the output stage, most of them use time-based loss functions [38, 66], which is consistent with the temporal coding. However, loss functions not belonging to the temporal coding family can also successfully train SNNs with time-based gradients [67, 69]. This phenomenon has not yet been clearly explained.
In this work, we investigate the relationship between rate coding and temporal coding by analyzing the loss functions that correspond to both coding schemes applied in time-based training algorithms. Besides, we study how to improve the performance of time-based training methods with appropriate loss functions (as well as other mechanisms). Our main contributions can be summarized as follows:
1. We prove that rate-based loss functions, extensively used in activation-based SNN training, suit the more biologically plausible time-based training scheme. This result implies that there are implicit relations between rate coding and temporal coding in SNNs.
2. We find that loss functions providing adequate positive overall gradients are more suitable for time-based training schemes in SNN training. Based on this, we propose the enhanced counting loss to improve the previously used mean square counting loss.
3. We transfer the training of scale factor used by weight standardization into thresholds. Experiments have shown that this action improves network performance.
4. We test our method on MNIST, Fashion-MNIST, NMNIST, CIFAR10, CIFAR100, DVS-Gesture, and CIFAR10-DVS datasets. The proposed method achieves SOTA performance around time-based SNN training methods on most datasets.
Neuron coding scheme. Spike timing information provides SNNs with richer coding schemes than non-spiking ANNs, which can be grouped into three main types: rate, temporal, and burst. Rate coding encodes information by firing rates of neurons [57], which is widely used in SNN training [7, 40]. Temporal coding is shown to convey richer and more precise information than rate coding [35, 53, 54], which can be classified into time-to-first-spike (TTFS) [55], rank-order [56], and phase coding [26]. Among them, the most commonly used scheme is the TTFS coding [3, 39, 43, 67], which can be applied to both the network input and neuron output. When applied to network input, it converts static pixel intensity into spike firing time (higher intensity corresponds to earlier input spike time) [66, 68]. When encoding the neuron outputs, it takes the timing of the first spike a neuron fires as the feature of a neuron [3, 68]. Besides, several recent works explore burst coding, a group of short inter-spike interval spikes, to improve the efficiency of information transmission and enhance the performance of SNNs [42]. Some works also extend the temporal-based rank-order coding into ANNs [25], bringing forward the time of decisions of ANNs without trading off accuracy.
Training methods of SNNs. SNN learning methods mainly include ANN-SNN conversion and direct training. ANN-SNN conversion assists SNN training with ANNs, which trains an ANN and replaces its activation functions by spiking neurons with transformed weights to get an SNN [7]. The converted SNN can achieve comparable performance as the source ANN when simulated long enough [5, 12, 21, 33, 48], whereas degrading seriously when the number of time steps is small [6]. Direct training mainly falls into two categories. The first category is activation-based training, which regards SNNs as RNNs and trains them with the Backpropagation Through Time (BPTT) method [11, 13-16, 20, 34, 36, 59, 60, 64]. Surrogate gradients [40] are used in this scheme to handle the discontinuity when neurons emit spikes. SNNs trained in this fashion can be scaled to large datasets like ImageNet [14] with low latency. They can deal with temporal tasks such as classifying neuromorphic datasets [11] while relying on rate coding: The loss function and gradient propagation scheme modify synaptic weights by modifying the firing rate of neurons. Specifically, the loss functions they use includes spike-train-based losses [50, 64], spike-count-based losses [59, 60], and those defined on discrete time steps [11, 15], among which losses defined on discrete time steps empirically perform better. The second category is time-based training, which takes the spike firing times as the intermediate variable to update synaptic weights [3]. Most algorithms in this track approximate the derivative of output spike timing to membrane potential  $\frac{dt_{out}}{du}$  as the negative inverse of membrane potential's time derivative  $\frac{-1}{du/dt}$  [3, 4, 38, 66, 67, 69]. Some works use other approximations [27], while others directly derive the relation between output spike firing times and input spike firing times [8, 39]. Although this training scheme has not yet trained SNNs with comparable performance as BPTT-based approaches, they satisfy the asynchronous nature of SNNs [69] and better utilize temporal coding in SNNs. In addition to the spike timing of neurons used in the backward calculation, the loss functions adopted in this scheme are typically time-based losses [3, 38, 39, 66]. However, other loss functions can also successfully train SNNs with time-based gradients [67, 69]. This leaves a question: Why can rate-coding-based losses successfully be applied in the time-based training scheme? We will analyze this question in the following content.
Spike response model. In this work, we use the spike response model for the neurons in the Spiking Neural Network. The state of neurons can be described by the following equations [3, 18, 67]:
$$
u _ {i} ^ {(n)} (t) = \int_ {t _ {i, \text {l a s t}} ^ {(n)}} ^ {t} \left(\sum_ {j} w _ {i j} ^ {(n)} \cdot s _ {j} ^ {(n - 1)} (\tau)\right) \cdot \epsilon (t - \tau) d \tau , \tag {1}
$$
$$
s _ {i} ^ {(n)} (t) = \delta \left(u _ {i} ^ {(n)} (t) - \theta\right), \tag {2}
$$
where  $u_{i}^{(n)}(t)$  denotes the membrane potential of neuron  $i$  in layer  $n$  at time  $t$ ,  $w_{ij}^{(n)}$  denotes the weight between neuron  $j$  in layer  $n - 1$  and neuron  $i$  in layer  $n$ .  $t_{i,\mathrm{last}}^{(n)}$  is the time of last spike of neuron  $i$  in layer  $n$ , and  $s_{i}^{(n)}(t)$  represents the spike emitted from neuron  $i$  at time  $t$ . The function  $\delta(\cdot)$  is the Dirac Delta function and  $\theta$  is the firing threshold. The spike response kernel is  $\epsilon(t) = \frac{\tau_m}{\tau_m - \tau_s} (e^{-\frac{t}{\tau_m}} - e^{-\frac{t}{\tau_s}})$ , where  $\tau_m$  and  $\tau_s$  are membrane and synapse time constants, respectively. Following [67, 69], we eliminate the influence of input spikes prior to the last output spike on membrane potentials.
Backpropagation based on spike timing. The backpropagation formulas in the time-based training scheme are given by [69]:
$$
\frac {\partial L}{\partial w _ {i j} ^ {(n)}} = \sum_ {i, t _ {k} \left(s _ {i} ^ {(n)}\right)} \frac {\partial L}{\partial t _ {k} \left(s _ {i} ^ {(n)}\right)} \frac {\partial t _ {k} \left(s _ {i} ^ {(n)}\right)}{\partial u _ {i} ^ {(n)} \left(t _ {k}\right)} \frac {\partial u _ {i} ^ {(n)} \left(t _ {k}\right)}{\partial w _ {i j} ^ {(n)}}, \tag {3}
$$
$$
\frac {\partial L}{\partial t _ {m} \left(s _ {j} ^ {(n - 1)}\right)} = \sum_ {i, t _ {k} \left(s _ {i} ^ {(n)}\right)} \frac {\partial L}{\partial t _ {k} \left(s _ {i} ^ {(n)}\right)} \frac {\partial t _ {k} \left(s _ {i} ^ {(n)}\right)}{\partial u _ {i} ^ {(n)} \left(t _ {k}\right)} \frac {\partial u _ {i} ^ {(n)} \left(t _ {k}\right)}{\partial t _ {m} \left(s _ {j} ^ {(n - 1)}\right)}, \tag {4}
$$
where  $L$  is the loss function,  $t_k(s_i^{(n)})$  and  $t_m(s_j^{(n-1)})$  are timings of spikes  $s_i^{(n)}$  (by neuron  $i$  in layer  $n$ ) and  $s_j^{(n-1)}$  (by neuron  $j$  in layer  $n-1$ ),  $u_i^{(n)}(t_k)$  is the membrane potential of neuron  $i$  in layer  $n$  at time  $t_k$ , and  $w_{ij}^{(n)}$  is the synaptic weight between neuron  $i$  in layer  $n$  and neuron  $j$  in layer  $n-1$ .
Figure 1: Forward and backward processes in activation-based training schemes and time-based training schemes. (a) Forward and (b) backward propagation in activation-based schemes: In the backward stage, the loss function updates parameters to adjust the firing rate. (c) Forward and (d) backward propagation in time-based schemes: In both forward and backward stages, information is delivered by spikes, which accords with the asynchronous nature of SNNs. The parameters are updated to adjust the firing times in the backward stage.
Figure 1(c)(d) shows the gradient propagation path in Eq. (3) and (4). It starts from the loss function and propagates to spike firing times  $t_k(s_i)$  of neurons, neuron membrane potentials at spike times  $u_i(t_k)$ , and synaptic weights  $w_{ij}$  in each layer. There are three major derivatives: output spike timing to membrane potential  $\frac{\partial t_k(s_i^n)}{\partial u_i^n(t_k)}$ , membrane potential to input spike timing  $\frac{\partial u_i^n(t_k)}{\partial t_k(s_j^{n-1})}$ , and membrane potential to weight  $\frac{\partial u_i^n(t_k)}{\partial w_{ij}}$ . The latter two can be directly derived since the original function (Eq. (1)) is continuous. The first derivative needs to take an approximation due to discontinuity at spike time:
$$
\frac {\partial t _ {k} \left(s _ {i} ^ {n}\right)}{\partial u _ {i} ^ {n} \left(t _ {k}\right)} = \frac {- 1}{d u _ {i} ^ {n} \left(t _ {k}\right) / d t}, \tag {5}
$$
which is first proposed by Spikeprop [3] and followed by many other works [38, 67, 69]. It has been proved that the sum of time-based gradients does not change between layers during backpropagation based on the above gradient calculation approach [69]:
$$
\sum_ {i} \sum_ {t _ {k}} \frac {\partial L}{\partial t _ {k} \left(s _ {i} ^ {(n)}\right)} = \sum_ {j} \sum_ {t _ {m}} \frac {\partial L}{\partial t _ {m} \left(s _ {j} ^ {(n - 1)}\right)}. \tag {6}
$$
Therefore, for time-based gradients, the derivative of spike firing time with respect to the loss function is equipped with more useful properties than the loss function itself. In other words, it is natural to view the loss function from the aspect of the gradients on spike firing times.
Here we first establish a framework to relate the gradients of the spike train to spike firing time for the output layer in Sect. 4.1 and 4.2. Starting with this framework, we will show how to view rate-based losses (defined on spike trains) as time-based losses (defined on spike firing times) with two examples, the rate-based counting loss used by [69] and spike train loss used by [67]. Sect. 4.3 analyzes the influence of overall gradients on spike timing on the SNN performance. Based on our analysis, we infer that loss functions providing adequate positive overall gradients are more suitable for time-based training schemes. Formulated by this inference, we propose the enhanced counting loss in Sect. 4.4. In Sect. 4.5, we propose to transfer the training of scale factor used by weight standardization into thresholds since threshold values do not appear in backpropagation formulas. At last, we will provide an overall algorithm description in Sect. 4.6.
4.1 Relate Gradients of Spike Train to Spike Firing Time
Unlike time-based losses defined on spike firing times, a rate-based loss  $L$  is defined on the output spike train  $s$  instead of the firing times of the output spike train  $t(s)$ . As a result, the spike train itself
should be an intermediate variable to pass the gradients, and a derivative from  $s$  to  $t(s)$  should be defined in the backpropagation stage:
$$
\frac {\partial L}{\partial t (s)} = \frac {\partial L}{\partial s} \cdot \frac {\partial s}{\partial t (s)}. \tag {7}
$$
Note that  $\frac{\partial L}{\partial s}$  is defined by the loss function itself, but  $\frac{\partial s}{\partial t(s)}$  is somewhat custom. Hence we add limitations on the derivative  $\frac{\partial s}{\partial t(s)}$ : First, gradients should not propagate from one spike to the timing of another spike, which means  $\frac{\partial \int s_i(t)}{\partial t'(s_{i'})} \neq 0$  only if  $i = i'$  and  $t = t'$ . Here  $\int s_i(t)$  means converting an infinite Dirac delta spike  $s_i(t)$  to a finite value 1. Hence the derivative of the timing of one spike is
$$
\frac {\partial L}{\partial t _ {k} \left(s _ {i}\right)} = \frac {\partial L}{\partial \int s _ {i} \left(t _ {k}\right)} \cdot \frac {\partial \int s _ {i} \left(t _ {k}\right)}{\partial t _ {k} \left(s _ {i}\right)}. \tag {8}
$$
Second, noticing there are still two degrees of freedom  $\left(\frac{\partial L}{\partial\int s_i(t_k)}\right.$  and  $\left.\frac{\partial\int s_i(t_k)}{\partial t_k(s_i)}\right)$  when calculating the gradient, we can control one of them to set  $\frac{\partial\int s_i(t_k)}{\partial t_k(s_i)} = 1$
However, for ease of understanding, we will allow  $\frac{\partial\int s_i(t_k)}{\partial t_k(s_i)}$  to take value  $-1$  in Sect. 4.2 and 4.4.
4.2 Why Can Rate-coded Losses be Applied in The Time-based Training Scheme?
Actually, rate-based loss functions have their time-based counterparts. For rate-based loss  $L$ , there is a time-based loss  $\mathcal{L}$  that has the same derivation, that is,  $\frac{\partial\mathcal{L}}{\partial t_k(s_i)} = \frac{\partial L}{\partial t_k(s_i)}$ . We first consider the case when loss functions do not involve spike timing in their expressions and have the following theorem:
Theorem 4.1. For a rate-based loss function  $L(\boldsymbol{s})$  which does not involve spike timing, with a spike-to-timing derivative  $\frac{\partial\boldsymbol{s}}{\partial\boldsymbol{t}(\boldsymbol{s})} = \mathrm{diag}\left(\frac{\partial\int s_i(t_k)}{\partial t_k(s_i)}\right) = I$  where  $(s_i,t_k)$  takes all neurons and spikes, it has an equivalent form  $\mathcal{L}(\boldsymbol{t}(\boldsymbol{s})) = \sum_{i}\sum_{t_k(s_i)}\frac{\partial L}{\partial s_i(t_k)}\cdot t_k(s_i)$  defined on the timing of spikes.
The detailed proof is provided in the Appendix. With Theorem 4.1, we can analyze why rate-based losses can be applied in the time-based training scheme.
Counting loss. The counting loss is commonly used for activation-based gradients [59, 60] satisfies the condition in Theorem 4.1. It has the form:
$$
L (\boldsymbol {s}, \mathbf {t a r g e t}) = \lambda \sum_ {i = 1} ^ {C} \left(\int_ {0} ^ {T} s _ {i} (t) d t - \operatorname {t a r g e t} _ {i}\right) ^ {2}. \tag {9}
$$
Here  $\lambda$  is a scaling constant, and  $C$  is the number of classes (which equals the number of output neurons). Besides, target is the target spike count sequence, and  $\mathrm{target}_i$  is the target spike number fired by neuron  $i$ . Typically,  $\mathrm{target}_i$  is bigger when  $i$  corresponds to the label. According to Theorem 4.1, the equivalent time-based loss function of the counting loss (when all  $\frac{\partial\int s_i(t_k)}{\partial t_k(s_i)} = 1$ ) is
$$
\mathcal {L} (\boldsymbol {t} (\boldsymbol {s})) = \sum_ {i} \sum_ {t _ {k} (s _ {i})} \frac {\partial L}{\partial \int s _ {i} (t _ {k})} \cdot t _ {k} (s _ {i}) = \sum_ {i} \sum_ {t _ {k} (s _ {i})} 2 \lambda \left(\int_ {0} ^ {T} s _ {i} (t) d t - \operatorname {t a r g e t} _ {i}\right) \cdot t _ {k} (s _ {i}). \tag {10}
$$
The detailed deduction is provided in the Appendix. The equivalent temporal-coded loss  $\mathcal{L}$  is linear for each spike firing time  $t_k(s_i)$ . It encourages a spike to fire earlier when the number of fired spikes of the corresponding neuron exceeds the target ( $\frac{\partial\mathcal{L}}{\partial t_k(s_i)} > 0$  when  $\int_0^T s_i(t)dt - \mathrm{target}_i > 0$ ) and fire later when the spike count is below the target. This is actually contrary to our expectations. The simplest way to fix it is modifying  $\frac{\partial\int s_i(t_k)}{\partial t_k(s_i)} = 1$  to  $\frac{\partial\int s_i(t_k)}{\partial t_k(s_i)} = -1$ , which is adopted by [69]. As a result, our analysis framework not only explains why counting losses works for time-based training but also shows why  $\frac{\partial\int s_i(t_k)}{\partial t_k(s_i)} = -1$  is necessary.
Spike train difference loss. The difference between spike trains [46] is also widely used in activation-based SNNs [29, 50, 64]. It can be described by the following formula:
$$
L (\boldsymbol {s}, \boldsymbol {s} ^ {\text {t a r g e t}}) = \lambda \sum_ {i = 1} ^ {C} \int_ {0} ^ {T} \left(\kappa * \left(s _ {i} - s _ {i} ^ {\text {t a r g e t}}\right) (t)\right) ^ {2} d t. \tag {11}
$$
Figure 2: The spike train difference loss. It aims to minimize the difference (purple area) between the (filtered) output spike train (red curve) and the target spike train (dashed curve). The kernel here is  $\kappa(t) = \alpha \cdot e^{-\alpha t}$ . (a)(b)(c)(d) are four different situations, where the output is pushed upward in (a)(c) and downward in (b)(d) by the loss. Note that the desired output spike train is empty in (d).
Here,  $s^{\mathrm{target}}$  is the target spike train for output neurons, and  $s_i^{\mathrm{target}}$  is the target spike train of neuron  $i$ . Typically the target spike train of the neuron corresponding to the label contains more spikes than other neurons.  $\kappa$  is a 1-d convolution kernel to smooth the spike trains. Theorem 4.1 can't directly apply due to spike timings being involved in the loss function, making it difficult to find its corresponding temporal-coded function. Instead, we apply qualitative analysis to it.
The spike train difference loss measures the height difference between two filtered spike trains, as shown in Figure 2. Sometimes we can find inconsistencies for it in the time-based training scheme: In both Figure 2 (a)(b), the spike should be moved earlier on the time axis. However, spikes will be moved in different directions (minimizing the purple area by moving up or down since it acts on spike scale  $s_i(t_k)$ ) in two cases under this loss. This spike direction inconsistency will impede training. On the other hand, if the target spike train is dense enough and covers all time ranges (Figure 2 (c)), the spike will always be pushed upward when the output spike train is sparse. Meanwhile, it is clear that output spikes of a neuron will be pushed down when the target spike train is empty (Figure 2 (d)). In both cases (c)(d), there are no inconsistencies. As a result, we can also correspond turning spike scale up with moving spike time earlier and let  $\frac{\partial\int s_i(t_k)}{\partial t_k(s_i)} = -1$  in this case. Our analysis explains why spike train difference loss works for time-based training and why  $\frac{\partial\int s_i(t_k)}{\partial t_k(s_i)} = -1$  is adopted in [67].
4.3 Balance Positive and Negative Time-based Gradient
According to Eq. (6), the positive part and the negative part of time-based gradients should be balanced to control the sum of gradients since they keep unchanged among layers.
Deficiencies of counting loss. The derivative  $\frac{\partial\int s_i(t_k)}{\partial t_k(s_i)} = -1$  will cause the sum of gradients on spike timing to comprise too much positive portion near convergence, especially when the target number of spikes for label neurons is large. To see this, according to Eq. (8), supposing a label output neuron emits  $m$  spikes, then the total gradients on these spikes are:
$$
\sum_ {s _ {i} \left(t _ {k}\right) | i = \text {l a b e l}} \frac {\partial L}{\partial t _ {k} \left(s _ {i}\right)} = m \frac {\partial L}{\partial t _ {k _ {0}} \left(s _ {i}\right)} = 2 \lambda m \left(\operatorname {t a r g e t} _ {\text {l a b e l}} - m\right). \tag {12}
$$