Paper: A Progressive Training Framework for Spiking Neural Networks with Learnable Multi-hierarchical Model (ICLR-2024)

Reviewer comment (Writing):
Please provide a more comprehensive explanation and analysis of the radical version of the LM-H model, and discuss its advantages compared to the vanilla LM-H model.

Author response:
As shown in Eq.6, the overall proportion term for historical information is represented as $(\mu_D)^{t-k}$, indicating that the extracted proportions at different time-steps follow an exponential distribution. In contrast, in Eq.7, the corresponding term changes to $\prod_{j=k+1}^{t} \mu_{D,j}$, allowing for more freedom and randomness in the extracted proportions. In addition, in the radical version, relevant factors that regulate the membrane potential ($\mu_{S,t}, \lambda_{S,t}$) and input current ($\lambda_{D,t}$) are independently assigned at each time-step, making the information representation range of the radical version wider than that of the vanilla version. We have added more details of the comparsion in Section A.5 of the revised version.

Reviewer comment (Presentation):
The authors have illustrated the distinction between their work and that of TC-LIF in the related works section. I suggest adding it into the introduction as well.

Author response:
Thanks for pointing it out. We have added the relevant content in the introduction of our revised version.

Reviewer comment (Novelty):
While the proposed model and training algorithm are unique and original, the paper could benefit from a more detailed discussion of why they propose the new model, how they differ from existing methods, and what specific contributions they make to the field of Spiking Neural Networks.

Author response:
Thanks for your valuable feedback! We would like to clarify that our motivation for proposing the LM-H model primarily stems from the limitations of the existing LIF model in addressing deep gradient computation and global information extraction challenges. While other advanced models such as GLIF and TC-LIF have been proposed by researchers in the SNN community, we believe that the LM-H model offers four distinct contributions that previous works have not encompassed:

(1) **The relationship between the LM-H model and the LIF model is elucidated through rigorous mathematical analysis, highlighting the superiority of the LM-H model.** We demonstrate that the LM-H model encompasses the LIF model as a special case (Eq.5-Eq.6), while also showcasing the superior information extraction capabilities of the LM-H model. This analysis ensures that the theoretical performance of the LM-H model is at least on par with the LIF model, as acknowledged by Reviewer LLZy.


(2) **The LM-H model possesses a remarkable capability to dynamically and flexibly extract global information across different time steps**. Through detailed analysis, we identify the specific functions of membrane-related parameters and set appropriate learning ranges for them. This allows the LM-H model at the $t$-th timestep to effectively incorporate relevant information, including input current and membrane potential from previous $t-1$ steps. This capability to consider information from multiple time steps simultaneously sets the LM-H model apart from other advanced models.

(3) **The LM-H model demonstrates superior scalability and transferability.** From a computational perspective, the LM-H model is a more general form of the LIF model. In terms of neuronal structure, the LM-H model incorporates a dendrite layer to store historical information for each time-step. When $\mu_D=\mu_S=0, \lambda_D=1$, the dendrite layers become invisible, and the LM-H model degenerates into a single-layer state consistent with the LIF model. These two aspects establish a close relationship between the LM-H model and the LIF model, facilitating easy migration of various training frameworks associated with the LIF model (such as hybrid training and online training) to the LM-H model.


(4) **The progressive STBP training framework is novel.** This novel training framework starts with the parameter settings of the LIF model as the initial state and dynamically optimizes membrane-related parameters during the training process to achieve the optimal state of the LM-H model. This design concept of "from special to general" allows for efficient and effective training of the LM-H model.

Reviewer comment (Theory):
What are the significant deficiencies in deep layer gradient calculation and capturing global information on the time dimension, as mentioned in this paper?

Author response:
In terms of capturing global information, the LIF model is limited compared to the LM-H model, as depicted in Eq.6. The LIF model can only consider the second term of the equation, which represents the current representation. This limitation arises from the nature of spiking neurons (LIF model), which can transmit binary information only once per time-step. If the membrane potential and spike firing information from different time-steps are not comprehensively utilized, the representation capability of the output spike sequence becomes restricted. Consequently, this limitation can have an impact on the performance of SNNs. Table 1 in this paper, along with Table R1, can demonstrate the significant performance gap between the vanilla LIF model and our LM-H model, particularly when applied to large-scale datasets.

Reviewer comment (Experiments):
The authors listed results of cifar data set on resnet, can you provide results on vgg as well?

Author response:
Thanks for your suggestion! We have conducted additional experiments using the VGG-13 architecture on the CIFAR-10 and CIFAR-100 datasets. As shown in Tab. R4, our results demonstrate superior performance compared to previous SOTA methods, even when employing smaller network structures and fewer time-steps. We have also included these content in the Appendix A.6 of our submitted version.

**Table R4: Comparison with previous SOTA works on CIFAR datasets, VGG structure.**
| Dataset     | Method | Architecture | Time-steps | Accuracy(%) |
| ----------- | ------ | ------------ | ---------- | ----------- |
|CIFAR-10|Diet-SNN [1]|VGG-16|5|92.70|
|CIFAR-10|Real Spike [2]|VGG-16|5, 10|92.90, 93.58|
|**CIFAR-10**|**Ours**|**VGG-13**|**4**|**93.98**|
|**CIFAR-10**|**Ours(radical version)**|**VGG-13**|**4**|**94.80**|
|CIFAR-100|Diet-SNN [1]|VGG-16|5|69.67|
|CIFAR-100|RecDis-SNN [3]|VGG-16|5|69.88|
|CIFAR-100|Real Spike [2]|VGG-16|5, 10|70.62, 71.17|
|**CIFAR-100**|**Ours**|**VGG-13**|**4**|**73.99**|
|**CIFAR-100**|**Ours(radical version)**|**VGG-13**|**4**|**74.79**|

[1] Nitin Rathi and Kaushik Roy. DIET-SNN: A low-latency spiking neural network with direct input encoding and leakage and threshold optimization. IEEE Transactions on Neural Networks and Learning Systems, pp. 1–9, 2021.

[2] Yufei Guo, Liwen Zhang, Yuanpei Chen, Xinyi Tong, Xiaode Liu, YingLei Wang, Xuhui Huang, Zhe Ma. Real spike: Learning real-valued spikes for spiking neural networks. European Conference on Computer Vision. 2022.

[3] Yufei Guo, Xinyi Tong, Yuanpei Chen, Liwen Zhang, Xiaode Liu, Zhe Ma, and Xuhui Huang. RecDis-SNN: Rectifying membrane potential distribution for directly training spiking neural networks. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 326–335, 2022.

Reviewer comment (Evaluation):
Both ImageNet and CIFAR datasets focus on static image classification, whereas the LH-M model incorporates historical data and demonstrates superior performance. Could you elucidate the underlying reasons for this enhanced performance in the context of LH-M's utilization of historical information?

Author response:
Thanks for your insightful comment! While ImageNet and CIFAR are static datasets, our experimental process, as consistent with previous works, involves inputting each image for multiple time-steps. Since spiking neurons (model) can only emit binary spikes (either firing or not firing) at each time-step, the representation range of information transmitted at a single time-step is inherently limited. Therefore, in order to enhance the representation ability of spiking neurons, it becomes necessary to consider the membrane potential ($\forall j\in[1, t-1], \boldsymbol{v}^l[j]$) and spike firing status ($\forall j\in[1, t-1], \boldsymbol{s}^l[j]$) at each previous time-step when deciding whether to fire a spike at the current time-step. This utilization of historical information allows for a comprehensive decision-making process. In fact, regardless of the input data type (static or neuromorphic), as long as we adopt multi-step training, the LM-H model can observe a broader range of information compared to the LIF model, thereby demonstrating its superior performance, as illustrated in Eq.6 in the main text.

Reviewer comment (Reproducibility):
How was the conversion from Artificial Neural Networks (ANN) to Spiking Neural Networks (SNN) executed, given that ANNs do not inherently incorporate temporal information? Considering the significance of historical data within the LH-M model, how was this temporal aspect effectively integrated during the conversion process?

Author response:
Thanks for pointing it out. We would like to clarify that our hybrid learning method can be divided into two stages. In the first stage, we adopt traditional ANN-SNN Conversion framework to obtain a LM-H model under the special case, which indeed does not contain temporal information. As shown in Eq.11 and its corresponding analysis, we derived that the most accurate mathematical mapping relationship between the converted LM-H model and the pre-trained ANN model (i.e., the minimum conversion error) is established under the condition $\mu_D=0,\lambda_D=1,\mu_S+\lambda_S=1$. Therefore, in the first stage of hybrid training, we replace the activation functions in the pre-trained ANN model with the LM-H models, adhering to the special case ($\mu_D=0,\lambda_D=1,\mu_S+\lambda_S=1$) layer by layer. 

However, in the second stage of the training process, all the membrane-related parameters of the LM-H model ($\mu_D,\lambda_D,\mu_S,\lambda_S$) are set as learnable parameters. Throughout the progressive training process, these parameters are dynamically optimized and updated to effectively extract temporal information. The detailed description and specific procedure of the algorithm can be found in Appendix A.4 of the revised version, where a comprehensive explanation is provided.

Reviewer comment (Evaluation):
ImageNet and CIFAR are all static image classification datasets, but LH-M involves historical information and performed better. Can you give an explaination?

Author response:
Thanks for your insightful comment! While ImageNet and CIFAR are static datasets, our experimental process, as consistent with previous works, involves inputting each image for multiple time-steps. Since spiking neurons (model) can only emit binary spikes (either firing or not firing) at each time-step, the representation range of information transmitted at a single time-step is inherently limited. Therefore, in order to enhance the representation ability of spiking neurons, it becomes necessary to consider the membrane potential ($\forall j\in[1, t-1], \boldsymbol{v}^l[j]$) and spike firing status ($\forall j\in[1, t-1], \boldsymbol{s}^l[j]$) at each previous time-step when deciding whether to fire a spike at the current time-step. This utilization of historical information allows for a comprehensive decision-making process. In fact, regardless of the input data type (static or neuromorphic), as long as we adopt multi-step training, the LM-H model can observe a broader range of information compared to the LIF model, thereby demonstrating its superior performance, as illustrated in Eq.6 in the main text.

Reviewer comment (Reproducibility):
How did you perform ANN2SNN conversion? ANN do not include temporal information, but in LH-M historical information is an important property.

Author response:
Thanks for pointing it out. We would like to clarify that our hybrid learning method can be divided into two stages. In the first stage, we adopt traditional ANN-SNN Conversion framework to obtain a LM-H model under the special case, which indeed does not contain temporal information. As shown in Eq.11 and its corresponding analysis, we derived that the most accurate mathematical mapping relationship between the converted LM-H model and the pre-trained ANN model (i.e., the minimum conversion error) is established under the condition $\mu_D=0,\lambda_D=1,\mu_S+\lambda_S=1$. Therefore, in the first stage of hybrid training, we replace the activation functions in the pre-trained ANN model with the LM-H models, adhering to the special case ($\mu_D=0,\lambda_D=1,\mu_S+\lambda_S=1$) layer by layer. 

However, in the second stage of the training process, all the membrane-related parameters of the LM-H model ($\mu_D,\lambda_D,\mu_S,\lambda_S$) are set as learnable parameters. Throughout the progressive training process, these parameters are dynamically optimized and updated to effectively extract temporal information. The detailed description and specific procedure of the algorithm can be found in Appendix A.4 of the revised version, where a comprehensive explanation is provided.