Paper: Temporal Flexibility in Spiking Neural Networks: A Novel Training Method for Enhanced Generalization Across Time Steps (ICLR-2024)

Reviewer comment (Presentation):
This paper is hard to follow due to the poor organization and presence. For instance, it would be helpful to clearly state the purpose of the experiments and analyses in Subsections 4.2 and 4.3.

Author response:
**Comment 1: This paper is hard to follow due to the poor organization and presence. For instance, it would be helpful to clearly state the purpose of the experiments and analyses in Subsections 4.2 and 4.3.**

Thank you for your kind reminder. We have added a summary at the beginning of Section 4 to provide a clear description of the purpose of all the experiments and analyses.

"In this section, we introduce our method Mixed Timestep Training (MTT), and its derivation process. We first experimented with a simple method, Naive Mixture Training (NMT), in Section 4.1 to bring up the temporal flexibility of SNNs. As a result, NMT not only succeeded in enabling SNNs to generalize across different time steps but also consistently outperformed the SDT-trained model at every single time step (even at $\mathrm{T}=\mathrm{T}_{max}$). In Section 4.2, we delve into the reasons behind NMT's performance improvement, examining both gradient and generalizability. Subsequently, in Section 4.3, we present our final method, MTT—an enhanced version of NMT that produces Temporal Flexible SNN (TFSNN) with high adaptiveness to time steps."

In addition to this, we have optimized the presentation and organization of the whole article. The texts highlighted in blue in our new version of the paper are our changes.

Reviewer comment (Writing):
The language used in the paper is quite loose, leading to some confusion. For example, when the paper states that "Increasing the time step (T) enhances the SNN’s expressiveness," it would be beneficial to provide a precise description of "expressiveness." Additionally, the claim that "increasing the time step leads to an increase in the calculated surrogate gradient error" requires further explanation. The treatment of time steps in this work deviates from typical learning methodologies for SNNs, potentially causing confusion for readers, as $T$ here does not represent the length of a spike sequence.

Author response:
**Comment 2: The language used in the paper is quite loose, leading to some confusion. For example, when the paper states that "Increasing the time step (T) enhances the SNN’s expressiveness," it would be beneficial to provide a precise description of "expressiveness." Additionally, the claim that "increasing the time step leads to an increase in the calculated surrogate gradient error" requires further explanation. The treatment of time steps in this work deviates from typical learning methodologies for SNNs, potentially confusing readers, as T here does not represent the length of a spike sequence.**

Thanks for your suggestion. We agree that we should add more explanation to avoid the confusion for reader. Therefore, we add additional descriptions of "expressiveness", "gradient error", and other related concepts in the revised manuscript. Hope our revision makes the context more clearly. 

We included the following sentences in Section 4:

"expressiveness" refers to the information encoding capability of the Spike trains. Due to the binary nature of the spike-based inter-layer information transmission in SNNs, their information encoding capacity is not as strong as that of ANNs. For example, a spike train composed of three spike values (each value can be 0 or 1) can only express 2^3 possible values. while the information encoding capacity of high-precision numerical data is almost unlimited (2^32). Fortunately, spike trains (SNN layer output) have both spatial and temporal dimensions, and expanding these dimensions can enhance the total information encoding capacity, i.e., "expressiveness". The spatial dimension is constrained by the network structure, while the temporal dimension is controlled by the SNN time step T. Increasing the time step T can lead to an improvement in the information encoding capacity, thereby enhancing the “expressiveness” of SNN.

"gradient accuracy" refers to the precision of the parameter gradients obtained by backpropagation. The reason for this notion arises from the fact that in the direct training of SNNs, the gradients computed by the backpropagation algorithm contain errors. These errors originate from the term $\frac{\partial s(i)}{\partial v(i)}$ in Eqn. 7, where this term is calculated as an approximated value by surrogate gradient during backpropagation because its original function is a step function. And there exists a certain error between the approximated value and the true value. As shown in Eqn. 7, as the time steps T increases, there are more instances of $\frac{\partial s(i)}{\partial v(i)}$, leading to a larger error in the calculated parameter gradient. [1] try to quantify the gradient error by the similarity between the gradient obtained through backpropagation (Eqn. 7) and the gradient computed by the finite differences (FDG).

Reviewer comment (Novelty):
The authors blur the concept of time steps, making it difficult for readers to distinguish existing work from traditional practices. Clearly, the authors seem to aim to bolster the significance of their paper with this assertion. However, it must be acknowledged that this approach is not in line with standard practices, particularly for SNNs. When employing this method, SNNs may not showcase the capability to outperform conventional neural networks. Instead, their representational capacity may be limited to a subset of RNNs.

Author response:
**Comment 2.1: The authors blur this concept here, making it difficult for readers to distinguish existing work from traditional practices. The authors seem to aim to bolster the significance of their paper with this assertion ... not in line with standard practices ... Instead, their representational capacity may be limited to a subset of RNNs.**

Thanks for mentioning these two valuable points. Due to the scope of the current work, we didn't discuss the meaning of time steps in SNN/TFSNN and its difference from the traditional practice of SNN setups. Regarding your concerns, we agree that we should add clarifications here. 

First want to state that we did not intend to blur any concepts here. The setup of using T as the latency of the network is a widely acceptable setup in many existing works, such as [3, 4, 5, 6, 7, 8, 9]. Differentiating the operation time steps and spike sequence length allows SNN to deal with non-spike signals and extends the applicable fields of spiking neural networks. 

Second, regarding our contribution and its feasibility with SNN, we understand that different setups would lead to different requirements for the deployment. For fully asynchronous chips like Speck, the calculation process is almost the same as biological neurons where each of the neurons receives and releases spikes independently. Therefore, the concept of the time step doesn't exist on this kind of chip and the time step introduced during GPU training is merely for a discrete simulation of the real asynchronous on-chip scenario. For synchronous/semi-asynchronous chips, e.g. Loihi, the concept of time step does exist in the latter type of chip, which usually needs to be synchronized with a clock. The on-chip model on this kind of chip and the GPU model are theoretically the same.
Regarding the datasets commonly used for SNN benchmarking, there are three typical families, traditional static datasets (e.g. CIFAR10, CIFAR100, ImageNet), traditional temporal datasets (e.g. Google Speech Command), and event datasets (e.g. DVS-CIFAR10, N-MNIST, SHD). The event datasets are either converted from traditional ones or recorded by spiking sensors such as DVS cameras. Each input of an event dataset is an event stream consisting of tuples as $(t_i, x_i)$ where $t_i$ is the emission time of the event and $x_i$ is the input position of the event. 

Now, we demonstrate the advantages of our method under 3 common settings respectively.
1. Deployment on synchronous/semi-asynchronous chips with static datasets (e.g. CIFAR) is the most frequently discussed topic. An input from static datasets is repeated for $T$ times and then fed into an SNN operating at time step $T$. This is also an essential prerequisite for ANN-SNN conversion and dynamic inference time steps. Here, the significance of TFSNN lies in the training-free energy-performance balance and the performance boosts it brings when taking MTT as a training method. 
2. For synchronous/semi-asynchronous chips with temporal and event datasets (e.g. DVS-CIFAR10), the significance of TFSNN mainly lies in the performance boosts, with possible energy-performance balance on a few number of datasets (e.g. DVS datasets shown in the paper). Inputs from temporal and event datasets, unlike static ones, are first collected into $T$ time frames with the same length and then fed into the network. We found that MTT can provide SNNs operating at $\mathrm{T}_{max}$ with higher performance on temporal and sequential tasks after ensuring a non-increasing time step sequence relative to input distance during temporal configuration sampling in the training phase.
3. The asynchronous chips are usually connected after a spiking sensor (e.g. a DVS camera) and are often used for event datasets. The asynchronous chips are also where the real power of SNN is exhibited. When training SNNs on GPU, the event inputs are collected into $T$ time frames for simulating chips discretely. After training, the weights of the model are downloaded to the chip directly for asynchronous inference. While existing direct training methods are only for a specific $T$, our TFSNN is largely disentangled from the influence of $T$ and thus suffers little performance loss after being deployed (see our response to reviewer wYLe). 

Third, we have to clarify the performance of existing SNNs can hardly surpass their ANN counterparts. The real advantage of SNN is not its performance but its low latency and low energy consumption when running on neuromorphic chips. This mechanism inspired by biological neurons allows SNNs to operate at extremely low power. For example, the authors of [1] calculated the energy consumption for various SNNs. Notably, for a Spiking ResNet-101 operating at T=4, the dynamic energy consumption of a single $224\times 224$ ImageNet input is only 4.53mJ. In [2], the authors deployed their SNNs on Loihi and reported a running power of 0.81~0.82W in Table 4, which is about 50 times more efficient than GPU models.

Reviewer comment (Evaluation):
The title of the paper may be considered somewhat overstated. The claim of "temporal generalization" lacks substantial support throughout the paper. The assertion of theory verification in Section 5.3 requires clarification.

Author response:
**Comment 3: The title of the paper may be considered somewhat overstated. The claim of "temporal generalization" lacks substantial support throughout the paper. The assertion of theory verification in Section 5.3 requires clarification.**

Thank you for your questions, we would like to answer these two questions separately. 

I. Support for the claim of "temporal generalization"
Although many of our experiments are related to theory validation and further exploration of our method, we provided multiple experiments to support the temporal generalizability achieved by our method. We designed Section 5.2 for this validation, where we first directly tested the performance of TFSNN at different time steps across different datasets. The related results can be seen in Table 4. We also demonstrate our method's compatibility with dynamic inference time steps methods (e.g. SEENN) in Table 5 to indirectly show the temporal flexibility of our method. 
The most decisive result comes when we compare our method with ANN-SNN conversion SOTA in Table 6. Conversion is a well-known method that exhibits temporal flexibility at high timesteps (e.g. $\mathrm{T}\ge 64$) but is only suitable for static datasets (since they have to train an ANN with identical structure first) and fails to generalize across low time steps. We find our TFSNN not only exhibits competitive performance at high timesteps similar to conversion methods but also generalizes well under extremely low timesteps. In short, we highlight the strengths of our method in that
- Our method generalizes well across both high and low timesteps.
- Existing SOTA conversion-based methods require fine-tuning or time-steps-related ANN redesigning for a specific time step, while our TFSNN is a zero-shot method and doesn't need any extra fine-tuning when switching to a different timestep.
- Conversion methods are only applicable to static datasets, while our MTT method can handle both static, temporal, and event datasets. 

II. The lack of clarification in Section 5.3
We appreciate your suggestion concerning Section 5.3. The original version indeed is somewhat confusing due to the excessively simple summary in the beginning and the lack of back-reference to former sections in the body part. We modified the summary to make it clearer and added essential references to the related part in the former sections.

A new summary at the beginning of Section 5.3 of the latest version: "In this part, we conduct validation experiments. We first verify our theory of gradient and generalization in section 4.2, and then perform TFSNN accuracy estimation described in section 4.3.3."

Reviewer comment (Novelty):
The temporal transformation module introduced in the paper bears similarity to existing concepts, such as multi-step RNNs. Providing a clear distinction or novelty in this module would enhance the paper's contribution.

Author response:
**Comment 4: The temporal transformation module introduced in the paper bears similarity to existing concepts, such as multi-step RNNs. Providing a clear distinction or novelty in this module would enhance the paper's contribution.**

Thank you for your valuable advice! If we understand correctly, multi-step RNN is the derivative of the concept of single-step RNN. A normal single-step RNN first receives an input $x_t$ at the beginning of each time step, then updates its hidden state $h_t$, and finally outputs $y_t$ at the end of the time step. A multi-step RNN (e.g. $k$ steps), however, consecutively receives inputs and updates the state for $k$ time steps before finally producing the output. 

We first distinguish between RNNs and SNNs. We understand the similarity between the GPU implementation of SNN and RNN, since SNN also receives a sequence of input at the beginning, then updates its membrane potential (similar to the hidden state), and eventually produces binary outputs at the end of each time step. Nevertheless, we still have to mention the obvious structural differences between an RNN layer and an SNN layer. First, after receiving the inputs, RNNs update their hidden state by a recurrent fully connected layer while the membrane potential update of every spiking neuron is independent of each other. Second, when stacking multiple RNN/SNN layers together, an RNN layer will directly receive the output (hidden states) of the former layer as the input, but two layers of SNN neurons are often connected with a fully connected layer or convolution layer. 

Next, we compare the difference between multi-step RNN and our proposed TTM. There are three major differences. 
- A multi-step RNN has hidden states while our TTM doesn't have any parameters or hidden states and only serves as a plug-in that groups and merges adjacent time steps.
- A multi-step RNN is equivalent to a single-step RNN when dropping the first $k-1$ outputs within every $k$ time step, while our TTM merges all the outputs within the same time step group.
- A multi-step RNN can only produce an output sequence shorter than the input, while our TTM can perform the transformation from a short sequence to a longer one.

We hope our response has answered your questions and look forward to hearing from you if you have further concerns!

Reviewer comment (Evaluation):
The neuromorphic hardware may not support different time steps for different blocks. For the discovered optimal combination of block time steps {3,2,2,3,5,3,6,2}, it may not be deployed for the real energy efficiency of SNNs. There can also be more discussion on the compatibility of the layer-wise adaption of time steps and the thought of (asynchronous) parallelism for neuromorphic computing.

Author response:
**Comment 1: The neuromorphic hardware may not support different time steps for different blocks. For the discovered optimal combination of block time steps {3,2,2,3,5,3,6,2}, it may not be deployed for the real energy efficiency of SNNs. There can also be more discussion on the compatibility of the layer-wise adaption of time steps and the thought of (asynchronous) parallelism for neuromorphic computing.**

Thank you for your feedback. Regrettably, current synchronous devices do not facilitate mixed timestep inference. Consequently, the identified combination cannot be deployed directly; we utilized it solely for exploratory purposes to gain valuable insights. However, since the performance of TFSNN is largely decoupled from the influence of the time step due to the design of MTT and the concept of timestep also doesn't exist on asynchronous chips, TFSNN is naturally more suitable for deployment on asynchronous SNN chips. To demonstrate this capability, we deployed the MTT-trained SNN on Synsense's Speck2e Devkit for asynchronous testing. Specifically, we trained two networks on the NMNIST dataset using SDT and MTT, respectively, and then deployed them on the Speck2e Devkit, as detailed in Table R1.

**Table R1 Test Accuracy on NMNIST(Adamw,lr=1e-3,batch_size=16,epochs=10)**
|Methods|GPU| Speck2e Devkit|
|-|-|:-:|
|SDT|98.61|96.18|
|MTT|97.79|98.5|

As a result, the SDT-trained model suffered a performance drop of about 2\% while the MTT-trained model maintained and even achieved a higher performance on Speck2e Devkit.

Reviewer comment (Theory):
Theoretical analysis is limited and there are many informal claims. For example, in Section 4.2, “we believe … balance between SNN expressiveness and gradient accuracy …” --- what is the formal definition of “expressiveness” and “gradient accuracy” and what’s the quantitative measurement of them? And “the new training loss may not converge, leading SNN to jump out of the local minimum point” --- to which loss the “local minimum” refers and why not converging can certainly lead to “jump out of the local minimum point” rather than move surrounding it? Is there any formal theoretical definition and analysis for these claims?

Author response:
**Comment 2: Theoretical analysis is limited and there are many informal claims. For example, in Section 4.2, “we believe … balance between SNN expressiveness and gradient accuracy …” --- what is the formal definition of “expressiveness” and “gradient accuracy” and what’s the quantitative measurement of them? And “the new training loss may not converge, leading SNN to jump out of the local minimum point” --- to which loss the “local minimum” refers and why not converging can certainly lead to “jump out of the local minimum point” rather than move surrounding it? Is there any formal theoretical definition and analysis for these claims?**

Thanks for your suggestion, indeed we have not provided clear definitions for the words “expressiveness” and “gradient accuracy” in the manuscripts. In the new version, we have included the following sentence to prove definitions for the two words (See the blue highlighted text in Sec 4.2 of the revised version):

“expressiveness” refers to the information encoding capability of the spike trains. Due to the binary nature of the spike-based inter-layer information transmission in SNNs, their information encoding capacity is not as strong as that of ANNs. For example, a spike train composed of three spike values (each value can be 0 or 1) can only express 2^3 possible values. while the information encoding capacity of high-precision numerical data is almost unlimited (2^32). Fortunately, spike trains (SNN layer output) have both spatial and temporal dimensions, and expanding these dimensions can enhance the total information encoding capacity, i.e., “expressiveness”. The spatial dimension is constrained by the network structure, while the temporal dimension is controlled by the SNN time step T. Increasing the time step T can lead to an improvement in the information encoding capacity, thereby enhancing the “expressiveness” of SNN.
“gradient accuracy” refers to the precision of the parameter gradients obtained by backpropagation. The reason for this notion arises from the fact that in the direct training of SNNs, the gradients computed by the backpropagation algorithm contain errors. These errors originate from the term $\frac{\partial s(i)}{\partial v(i)}$ in Eqn. 7, where this term is calculated as an approximated value by surrogate gradient during backpropagation because its original function is a step function. And there exists a certain error between the approximated value and the true value. As shown in Eqn. 7, as the time steps T increases, there are more instances of $\frac{\partial s(i)}{\partial v(i)}$, leading to a larger error in the calculated parameter gradient. [1] try to quantify the gradient error by the similarity between the gradient obtained through backpropagation (Eqn. 7) and the gradient computed by the finite differences (FDG).

Due to the multidimensional nature of neural networks, even on the same training dataset, variations in training methods, and initial random seeds, the same networks may converge to various distinct local minima, leading to divergent performances on test data. The introduction of the concept of loss landscape provides an intuitive way to assess these local minima. Generally, sharper local minima indicate lower generalization of the network, while flatter local minima indicate higher generalization of the network.
We think that “jump out of the local minimum point” and "moving surrounding it" do not contradict each other in the original sentence. They both refer to the idea of NMT making the SNN leave (Instead of falling into) a sharp local minimum. Eventually, the SNN reaches a flatter local minimum (See Fig. 9 in A.10), which ensures that even when changing the time step T, the SNN still cannot escape it.


[1] Yuhang Li, Yufei Guo, Shanghang Zhang, Shikuang Deng, Yongqing Hai, and Shi Gu. Differentiable spike: Rethinking gradient-descent for training spiking neural networks. In A. Beygelzimer,Y. Dauphin, P. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, 2021b. URL https://openreview.net/forum?id=H4e7mBnC9f0.