Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 2: Strong Reject: For instance, a paper with major technical flaws, and/or poor evaluation, limited impact, poor reproducibility and mostly unaddressed ethical considerations.
review_content: summary: The authors present a well-designed ANN-to-SNN conversion method. They claimed to discover and discuss the overlooked conversion errors: coding error and inference error. A QReLU activation for the training phase and a buffered none-leaky neuron for the inference phase were proposed. The authors gave good results on classification and detection tasks. To some extent, the authors did not conduct detailed literature research and over-claimed their contributions.
strengths_and_weaknesses: Strengths: 
1. This paper is easy to follow. The authors introduce the QReLU function, buffered none-leaky neurons to promote accuracy and integrate the shift-based multi-bits spike train to increase representation efficiency.
Weakness:
1. The major weakness is the novelty. The main contributions are the error identification and a quantization ReLU activation, both of which have been proposed in some previous work [1], [2]. Also, the negative spike has been proposed by spiking-yolo. Please clarify the difference, or is this work a combination of these techniques?
2. Experiments are not enough to demonstrate the effectiveness of the proposed method. For example, it is mentioned in line 177 that λ is learnable, according to Eqn. 9. Please clarify whether T needs to be set. If T needs to be set, what is it equal to?
3. The authors use a newly proposed neuron model which may not be supported by the current hardware, which will diminish the significance of this paper.
4. There are some ambiguous expressions in the paper that are hard to understand. 

[1] Yan, Z., Zhou, J., & Wong, W. F. (2021). Near lossless transfer learning for spiking neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence.
[2] Bu, T., Fang, W., Ding, J., Dai, P., Yu, Z., & Huang, T. (2022). Optimal ANN-SNN Conversion for High-accuracy and Ultra-low-latency Spiking Neural Networks. In International Conference on Learning Representations.
questions: 1.The idea of QReLU is similar to paper [1] & [2]. Please demonstrate the differences between them. Comparisons of the performance should also be included. 
2.V_(T_o) < 0 is equivalent to the over-fired case. My question is: Does this condition restrict that there is only one negative pulse before T_o? And, given that the final membrane potential may not exceed Vth, how can the effects of the over-fired situation be mitigated?
3.I don't understand how the experiment of spike camera detection is implemented. Please specify more details, including how long the sequence you input to the SNN and how the data used was obtained.
4. What does the time step of ANN in Fig.2 mean?
5. Line 205 says that the computational-expensive multiplication operation needs to be avoided. Please explain how log2 operation in the newly designed neuron will be implemented in hardware?
6. Line 197 says "we integrate shift-based multi-bits spike", I didn't find a statement about "shift" in the paper. What is "shift"?
7. There is no ablation study on “QReLU” and “Multiple Bits Spike Train” found in the main paper.
8. If λ is learnable, does T need to be pre-set in QReLU? Can the performance of different time steps be provided under the same model?
9. As the aim of this work is to get loss-free conversion, if the buffered potentials are the integration of all inputs across time, why not just output the buffered potential to get loss-free conversion.
limitations: This paper does not do enough quantitative analysis and explanation of the proposed negative spike method. Neurons that generate negative spikes belong to the special category of spiking neurons.
Since the authors use a new type of neuron, and the new neuron is obviously more complex than the traditional neuron, the authors should include more analysis about the power consumption and hardware implementation of this neuron.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: The paper describes a method to convert ANN with quantized ReLu activation functions into a spiking network emitting non-binary spike trains. The theory extends previous work on rate-based conversion from relu ANN to spiking networks: the authors characterize conversion error into sub-types. Beside the coding error (quantization error) they categorize the conversion error as under- and over-spike error and suggest methods to reduce them. One idea is to send signed spikes to signal when the accumulated inputs cross the threshold downward (effectively cancelling the previously transmitted spike and solving the over-spike error). The second is to transmit non-binary spikes with a spike amplitude quantized on a scale of powers of 2. It yields non-binary spiking networks with high-performance (for example 75% top1 accuracy on ImageNet  which is on par with current popular VGG or ResNet baselines). 
strengths_and_weaknesses: ## Strengths:
(1) The characterization of over- and under-fire conversion error is relevant and useful to design better conversion schemes of spiking networks to non-spiking networks.

(2) The performance is very high and it bridges the gap between spiking networks and regular quantized neural networks.

(3) The reduction of  over-fire error with negative spikes seems to have a large positive-effect on the accuracy and its description is understandable.

## Weaknesses:
(1) The paper is poorly written: most importantly the main text lacks of mathematical rigor and uses confusing notations. For instance in section 3.1, it is not clear what is a definition of a notation for $S^*(t)$. The closest to a definition would be equation (4) but this defines the sum over $t$ and not $S^*(t)$ per-se which is confusing. So the concepts of "ideal conversion" is defined in a cumbersome and complicated way although it seems quite important. Also the definitions of $r_i$,  $S^*(t)$ and $r^*_i$ are unclear and maybe redundant. Because of this I did not understand why a proof mentioned at line 135 is necessary, I read through the proof in appendix anyway and it seems to rely on some wrong arguments: for instance it is written in the appendix "For $V_a$ > 0 we infer $C$ > 0" but C depends on $t$ although $V_a$ is a quantity integrated overtime so $V_a > 0$ cannot imply $C > 0$. This statement is either wrong or unclear, although it is probably a minor argument for the proof, it shows the general lack of rigor in the theoretical section. I think this paper will be much better if the theory section is re-written, mostly sections 3.1, 3.4 and this proof which are hard to read and unclear at the moment. 

(2) The implementation is not clear and the code is not provided, this is a real problem because details are missing to fully understand the algorithm. Concretely I see two possible ways of implementing this model but I do not know which one is implemented. It is not the same if the layer 2 fires spikes S(t) right after layer 1 S(t) or whether layer 2 waits the end of the computation S(T) of layer 1 before computing S(1). 
- First possibility: the code can be something rather simple like replacing the ReLU or QReLU with a spiking version of it which receives the integrated sums of spikes and outputs the integrated sum of spikes, if this is what is implemented it should be acknowledged clearly and the the theoretical section should clearly acknowledge that the latency accumulates at each layer. Also, if this is the implementation the activity is buffered at each layer so the necessity of the additional spike buffer is probably unnecessary.

- Second possibility: the model is implemented as a sort of recurrent neural network where the spikes S(t) at all layers are computed before sending them to S(t+1). However, training a recurrent network in this way would require to use back propagation through time which typically multiply the memory consumption of the algorithm by a factor T and raises a clear drawback to train large ImageNet or VOC models on GPUs. Considering the trade off that is therefore necessary between T and batch size, it should at least be debated somewhere.

In any case, even if the implementation is done in another way which I did not describe here. This should be acknowledged and described clearly. Ideally the code should be shared so that any ambiguity can be lifted by reading the code directly.

(3) I think that the theoretical statement that the negative spike can correct the over-fire problem is only true with a single hidden layer. A clear counter example with two layers can be designed where a neuron in hidden layer 1 spikes positively at time step 1 (S(1)=1) which engages non linear computations in the hidden layer 2 at the same time step, and then spikes negatively at time step 2 (S(2)=-1) leading to a QReLU output of 0. Assuming the second possible implementation that I described in the paragraph above it is not guaranteed that the accumulated effects of the negative and positive spikes cancel out because of the non-linearity in layer 2. Unfortunately I cannot be sure whether this can occur in the simulation because of the missing information about the implementation. If it does occur, the imperfections of the negative spike solution of over-fire errors should be acknowledged in the main text.

(4) By introducing the concept of non-binary spike trains the conceptual boundary between spiking neurons and quantized ANN is blurred. This should be acknowledged clearly and the remaining difference and benefits of non-binary SNN and Q-ANN should be discussed. About their similarities, one could acknowledge:
- This conversion method requires a pre-training with a well chosen QReLU activation function so it requires the similar training algorithm as a quantized ANN (say a binary feed-forward neural network trained with straight through estimators). 
- Certainly most current hardware capable of simulating a non-binary SNN could also implement a Q-ANN. Which raises questions about the benefits of the spiking solution.

About there differences, I would love to see some quantitative comparisons, for instance:
- Can one add the Q-ANN into the energy ratio plot ? (not sure how to count integer multiplications into the plot relying on the number of additions, but this was done somehow to integrate spikes with amplitude larger than one in this plot already so why not?). If not, finding common metric to estimate to energy consumption of Q-ANN and non-spiking network would be great in this context but it might be left for future work.

(5) Details of implementation about the Yolo architecture are not provided. It probably requires to implement non-binary communications on some unusual blocks like residual blocks. Although a paper is cited for this purpose it is necessary to explain whether something similar or different is used. I also think that this can be problematic in regards to the theory section depending on the implementation, so a clear description of the implementation of residual block using the notations and concepts from the theory section would be strongly welcome. Again, one option is to publish the code.

questions: I wrote suggestions and questions in the Strengths and weaknesses section.

## Ethical concern:
The implementation of Yolo with DVS cameras is likely to converge towards low-consumption surveillance applications which raises ethical concerns. Although there are many applications of Yolo with DVS cameras which would not necessary lead there, the current Figure 4 demonstrates how to separate people from objects so it depicts quite explicitly how this technology can be applied for surveillance without mentioning the ethical concerns. There are many ways to reduce this negative impact, here are some concrete options: choose a different example in Figure 4 (and Figures from Section B in appendix) and publish the code so that independent researchers can understand better how to avoid misuses of the technology.
limitations: I described limitations in the weaknesses section.
review_title: null

final_score: rating: 3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.
review_content: summary: This paper proposes to improve the ANN-to-SNN conversion method by several techniques including QReLU activation, buffered non-leaky-integrate-and-fire neuron, and multiple bits spike train. Experiments on both classification and detection tasks demonstrate strong performance.
strengths_and_weaknesses: Strengths:
1. This paper not only considers the common classification task but also demonstrates strong results on the detection task.

Weakness:
1. The novelty of this paper is limited. The conversion error has already been theoretically analyzed in the literature [1,2,3,4]. Particularly, the inference error in this paper is studied as “unevenness error” in [3] and “deviation error” in [4] (while [4] is very recent and available after the submission deadline, [3] is earlier and should be noted). The quantized ReLU activation in this paper has been adopted in [3] as “quantization clip-floor activation”. The proposed buffered non-leaky IF neuron has been similarly adopted in the literature [5]. And the multiple bits spike train is inspired by existing literature except that this paper realizes it in a different way. So it looks like this paper simply combines multiple existing techniques and the novelty is very limited.

2. It is unclear how the proposed buffered neuron model and multiple bits spike train can be implemented on neuromorphic hardware. The compatibility with hardware should be discussed.

3. This paper does not use common neuron models and tries to enforce the inference of SNNs to be the same as quantized ANNs by proposing new neuron models. Then what is the advantage compared with directly applying well-established quantized ANNs that can also have hardware-efficient implementation? There should be some discussion and comparison. 

[1] Deng, S., & Gu, S. (2020, September). Optimal Conversion of Conventional Artificial Neural Networks to Spiking Neural Networks. In International Conference on Learning Representations.

[2] Li, Y., Deng, S., Dong, X., Gong, R., & Gu, S. (2021, July). A free lunch from ANN: Towards efficient, accurate spiking neural networks calibration. In International Conference on Machine Learning (pp. 6316-6325). PMLR.

[3] Bu, T., Fang, W., Ding, J., Dai, P., Yu, Z., & Huang, T. (2021, September). Optimal ANN-SNN Conversion for High-accuracy and Ultra-low-latency Spiking Neural Networks. In International Conference on Learning Representations.

[4] Meng, Q., Yan, S., Xiao, M., Wang, Y., Lin, Z., & Luo, Z. Q. (2022). Training much deeper spiking neural networks with a small number of time-steps. Neural Networks.

[5] Thiele, J. C., Bichler, O., & Dupret, A. (2019, September). SpikeGrad: An ANN-equivalent Computation Model for Implementing Backpropagation with Spikes. In International Conference on Learning Representations.

questions: See Weakness above.
limitations: There is no potential negative societal impact.
review_title: null

Meta-review: metareview: Spiking Neural Networks (SNNs) have some advantages, especially in terms of power consumption, over standard Artificial Neural Networks (ANNs). However, most trained networks are ANN and therefore this work presents a conversion scheme from SNNs to ANNs with some desired properties. Unfortunately, the reviewers found the contribution of this work insufficient in terms of novelty. The authors argued, in the rebuttal, that there are differences between prior art and the current paper. The reviewers acknowledged these differences but were not convinced that the differences are significant enough. The authors also noted that this work was sent to publication in Jan 2021 and was rejected while some of the relevant papers were published after this date. We understand the frustration that this situation is likely to generate. However, when reviewing this work the relevant date is the deadline for NeurIPS submission deadline and therefore these studies should be compared to in the paper.
confidence: Certain
Decision: Reject