Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper focusses on an important problem of improving supervised training of spiking neural nets (SNNs), as the current state of SNN training is lacking as compared to that for standard ANNs. The authors propose a couple simple but effective methods for improving both training efficiency and classification accuracy:  temporal efficient training (TET) which simply accumulates error gradients for each time step, and time inheritance training (TIT), which is a repeated training schedule with increasing simulation times.  In extensive experiments the authors demonstrate that their approach improves training time and accuracy. 
main_review: While the proposed improvements, TET and TIT, may appear only as minor tweaks as compared to previous work, the demonstrated improvements in speed and accuracy suggest that the paper may have substantial impact for SNN research community. 
summary_of_the_review: Efficient supervised learning of deep SNNs is a problem that has resisted research efforts for many decades, so even incremental improvements are welcome in this area. The results of the paper offer a much desired step towards better SNN backprop algorithms that promise to unleash the many advantages of SNNs.  
The paper is lucidly written and provided experiments strongly support the claims.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The authors propose the temporal efficient training (TET) method that features loss evaluation at every timestep. As such, the main difference between TET and SDT lies in the fact that TET evaluates loss at every timestep with the correct label vector whereas SDT evaluates loss at the last timestep. But unfortunately, this could not be done because of the use of $L_{TOTAL}=(1-\lambda)L_{TET}+\lambda L_{MSE}$, where $L_{MSE}$ is evaluated at the last moment. The proposed method is simple but its efficacy outperforms the SOTA results for various networks on various datasets.

main_review: $\textbf{Strengths:}$ 
The high classification accuracy on DVS-CIFAR10 is very impressive.

$\textbf{Weaknesses:}$
In spike of the excellent classification records achieved in this work, the authors’ work leaves a number of uncertainties that should be thoroughly addressed to not mislead the readers. My main concerns are as follows.

1.	I wonder if the use of TET is the main cause of the improvement in classification accuracy. The authors actually used $L_{TOTAL}=(1-\lambda)L_{TET}+\lambda L_{MSE}$ rather than $L_{TET}$. If the use of TET is the main cause, the comparable results should be achieved using solely $L_{TET}$. Given the use of $L_{TOTAL}$, the feature of TET such that loss evaluation at every timestep is undermined because $L_{MSE}$ is evaluated at the last timestep. Additionally, the loss landscape analysis is also undermined because the actually used loss is $L_{TOTAL}$ not $L_{TET}$. Therefore, the authors should clarify the effect of TET only if TET is the key to the present work.

2.	I wonder if the time inheritance training is the unique feature of TET. The authors should discuss if the same method can successfully apply to SDT as well. This method may be the main cause of the accuracy improvement. The authors should provide the general framework of the time inheritance training method to identify its scalability to different networks on different tasks rather than naively discussing about a single hypothetical case. 

3.	The authors repeatedly remarked the mismatch of gradient and loss for SDT. I wonder on what ground the mismatch is assured and how the use of TET can resolve the mismatch. Also, the point that “TET needs the second term $\partial O/\partial W$ close to 0 to make the $L_{TET}$ convergence.” lacks its theoretical grounds, which should be proven. If this is true, the accuracy should be independent of timestep length T for training; yet, Figure 4 identifies that this is not the case. Even if it is true, this point cannot directly explain the accuracy improvement because learning was done with $L_{TOTAL}$

4.	There are many statements that may mislead the readers.
“Another training issue in the memory and time consumption …” on Page 1. Please specify when this is the truth.
“Furthermore, since the TET applies optimization on each time point…” on Page 2. Is this true for the use of $L_{TOTAL}$?
“TET algorithm optimizes each moment’s output, enabling us to extent…” on Page 6. I am very puzzled about this statement. What did the authors mean by “naturally extend the simulation time”?

5. The improvement in classification accuracy on DVS-CIFAR10 is impressive. However, for the moment, it is unclear the direct cause of the improvement for the reason that I remarked above.

Minor concerns:
There are many statements that may mislead the readers.
“Another training issue in the memory and time consumption …” on Page 1. Please specify when this is the truth.
“Furthermore, since the TET applies optimization on each time point…” on Page 2. Is this true for the use of $L_{TOTAL}$?
“TET algorithm optimizes each moment’s output, enabling us to extent…” on Page 6. I am very puzzled about this statement. What did the authors mean by “naturally extend the simulation time”?
summary_of_the_review: Based on the main concerns listed above, I recommend for reject for the current form of manuscript.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposed a new loss function called TET for directly training SNNs.
main_review: Strength: The results are good compared to existing works. The paper also has a comprehensive comparison between the proposed TET and regular SDT.
Weakness: The proposed method only changes the loss function. The novelty may not be enough. It is still a kind of surrogate gradient method and cannot solve the approximation made by smoothing the gradient of the discontinuous spiking function.
summary_of_the_review: 1. Missing reference: There’re some recent papers that should be discussed in the related works, e.g. [1]
2. I'd like to learn more about the details of data preprocessing. For example, does the real value in CIFAR and ImageNET datasets directly used as the inputs? In DVS-CIFAR10, how does the resolution reduced? How is the time length shrunk to 10 time steps? 
3. Performance comparison: In CIFAR10, can the authors also provide the performance of the 5 layers CIFARNet for better comparison with existing works? In DVS-CIFAR10, do other works also use data augmentation, e.g.[2]? Does the network adopted in this paper have a similar number of parameters to the existing works in the comparison?



[1] Yang, Y., Zhang, W., & Li, P. (2021, July). Backpropagated Neighborhood Aggregation for Accurate Training of Spiking Neural Networks. In International Conference on Machine Learning (pp. 11852-11862). PMLR.
[2] Zhenzhi Wu, Hehui Zhang, Yihan Lin, Guoqi Li, Meng Wang, and Ye Tang. Liaf-net: Leaky integrate and analog fire network for lightweight and efficient spatiotemporal information processing. IEEE Transactions on Neural Networks and Learning Systems, 2021.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes a new training approach, the temporal efficient training (TET). This algorithm utilizes a new loss function to improve the generalizability of SNNs. Further, a new training pipeline is presented to reduce the simulation time of SNNs. This work outperforms the SOTA on the static datasets and neuromorphic datasets.
main_review: There are some main issues to be addressed:
1. The TET method and the TIT method can be viewed as two individual parts. The TET method is a new temporal loss function and the TIT is a fine-tune method. If so, an ablation study is required to show the contribution of these two parts in temporal efficiency individually. Furthermore, can the TIT method be applied to other SNNs as long as these SNNs are trained with temporal loss (e.g., the STCA loss [1])? 
2. In the section 4.3, this paper claims the TIT method could significantly reduce the training time. Here are some questions.
(1) the time spent per epoch is not linearly proportional to the simulation length, so it is inappropriate to claim ‘As a result, the TIT can reduce the training time cost by half.’
(2) based on the above analysis, actually, it seems difficult to select the training epoch and simulation length for the TIT method to achieve temporal efficiency for training SNNs.
3. In table 3, the network performance across static datasets and neuromorphic datasets, why the simulation length of two networks with the TET on the Imagenet are different? Is it from original paper or based on the optimization?
4. For time scalability, it’s better to supplement the result of relative growth rate for both SDT and TET. 
5. Some minor comments, such as the unclear figure legends, the missing curve of “Training from scratch” in Fig 4A, inconsistent decimal places of accuracies and so on.

[1] Gu, et al. STCA: Spatio-Temporal Credit Assignment with Delayed Feedback in Deep Spiking Neural Networks. IJCAI 2019. 


summary_of_the_review: The novelty is limited, as the core idea and the relative derivations are straightforward.
review_title: null

Meta-review: The paper proposes a new loss function for the training of spiking neural networks leading to significant improvements in generalization performance across a variety of datasets and network architectures. While conceptually simple, the approach leads to substantial performance gains, and some intuition is provided to explain its success.

The reviewers are split on the issue of significance of the paper, in part due to the simplicity of the proposed loss function. Still, good results speak for themselves, and the effectiveness of the technique has been demonstrated thoroughly.
Decision: Accept (Poster)