Canonic Signed Spike Coding for Efficient Spiking Neural Networks

Authors: Yiwen Gu, Junchuan Gu, Haibin Shen, Kejie Huang
Primary area: applications to neuroscience & cognitive science
Keywords: Spiking neural network, Spike coding scheme, Stepwise weighted spike

Abstract:
Spiking Neural Networks (SNNs) seek to mimic the spiking behavior of biological neurons and are expected to play a key role in the advancement of neural computing and artificial intelligence. The conversion of Artificial Neural Networks (ANNs) to SNNs is the most widely used training method, which ensures that the resulting SNNs perform comparably to ANNs on large-scale datasets. The efficiency of these conversion-based SNNs is often determined by the neural coding schemes. Current schemes typically use spike count or timing for encoding, which is linearly related to ANN activations and increases the required number of time steps. To address this limitation, we propose a novel Canonic Signed Spike (CSS) coding scheme. This method incorporates non-linearity into the encoding process by weighting spikes at each step of neural computation, thereby increasing the information encoded in spikes. We identify the temporal coupling phenomenon arising from weighted spikes and introduce negative spikes along with a Ternary Self-Amplifying (TSA) neuron model to mitigate the issue. A one-step silent period is implemented during neural computation, achieving high accuracy with low latency. We apply the proposed methods to directly convert full-precision ANNs and evaluate performance on CIFAR-10 and ImageNet datasets. Our experimental results demonstrate that the CSS coding scheme effectively compresses time steps for coding and reduces inference latency with minimal conversion loss.

Decision: reject
Reviewer scores: 6, 6, 6, 3

Reviews and discussion:
<thread>
creation date:
1730003935306

thread_type:
official_review

<thread_object>
creator: reviewer_dpat

<content>
summary:
This paper proposes a new ANN-SNN Conversion learning framework, which is based on Canonic Signed Spike (CSS) coding scheme and Ternary Self-Amplifying (TSA) neuron model.

strengths:
1. The authors conduct theoretical analysis and proof for the ANN-SNN Conversion process based on CSS coding scheme.

weaknesses:
1. The concept of firing negative spikes has already been proposed by [1].
2. This paper involves firing negative spikes, however, most of the comparative works selected by the authors in Table 2 are based on firing binary spikes. Furthermore, training high-performance converted SNN models under low time latency is currently one of the research focuses in the domain of ANN-SNN Conversion, but the authors do not reveal the performance of CSS under low time latency ($\leq 4$ time-steps) as demonstrated in [1, 2, 3].

[1] Li C, et al. Quantization framework for fast spiking neural networks. Frontiers in Neuroscience, 2022.

[2] Bu T, et al. Optimal ANN-SNN conversion for high-accuracy and ultra-low-latency spiking neural networks. ICLR, 2022.

[3] Hao Z, et al. Bridging the gap between ANNs and SNNs by calibrating offset spikes. ICLR, 2023.

========================

After reading the authors' further response, I will provide the following points of views for AC and other Reviewers to consider:

* The so-called "silent period" and "fire ternary spikes", as shown in Algorithm 1, can it be effectively implemented on neuromorphic hardware without causing additional time latency and energy consumption?
* Is it fair to directly compare CSS with other vanilla conversion methods, which are based on firing binary spikes and do not introduce special mechanisms (e.g. silent period) ?
* In Tab.3, COS [3] is a conversion method that emits spikes layer by layer, rather than a method that emits spikes step by step. But the authors directly calculate the time latency based on "$T+\rho\times(\textbf{Network Layers})$" time-steps, without respecting the $T+\rho$ statistical standard used in the original paper. Is this reasonable?

In their response, the authors think that my rating opinion is vague, so the above three points are the relatively detailed reasons I further provided.

questions:
See Weakness Section.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730434254461

thread_type:
official_review

<thread_object>
creator: reviewer_4dbk

<content>
summary:
The paper proposes a novel spiking input encoding called Canonic Signed Spike (CSS), which incorporates non-linearity into the encoding process by weighting spikes at each step of neural computation. To mitigate the temporal coupling phenomenon in the SNN using CSS encoding, the paper proposes a Ternary Self-Amplifying (TSA) neuron model and a one-step silent period, which reduces the encoding latency.

strengths:
1. The idea that decouples the input and output when inference SNN is novel.
2. The experiments are detailed and convincing.

weaknesses:
1. The method section is hard to read and some notations are not explained. For example, what are the detailed steps that generate Eq(5), and what is the meaning of τ (\tau) in Eq(5)? 
2. The compared SOTA work in rate code is not superior. Fast-SNN [1] and Offset[2] works can achieve lossless conversion accuracy with less latency (time-step).
3. The energy consumption of the SNN encoded by CSS is missing. 
I will raise my rating if the authors can solve the questions.

[1] Hu Y, Zheng Q, Jiang X, et al. Fast-SNN: fast spiking neural network by converting quantized ANN[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
[2] Hao Z, Ding J, Bu T, et al. Bridging the gap between anns and snns by calibrating offset spikes[J]. arXiv preprint arXiv:2302.10685, 2023.

questions:
1. Can the author provide a comparison between this work and other SOTA SNN conversion works such as (Fast-SNN[1], Offset[2])? 
2. Can the author provide the number of synaptic operations of the SNN using CSS? The calculation code can be found in https://github.com/zhouchenlin2096/Spikingformer/tree/master/energy_consumption_calculation.
3. Can the authors provide a more simple explanation of the equivalence of ANNs and SNNs?


[1] Hu Y, Zheng Q, Jiang X, et al. Fast-SNN: fast spiking neural network by converting quantized ANN[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
[2] Hao Z, Ding J, Bu T, et al. Bridging the gap between anns and snns by calibrating offset spikes[J]. arXiv prepri
</content>
</thread_object>

</thread>
<thread>
creation date:
1730438038417

thread_type:
official_review

<thread_object>
creator: reviewer_adde

<content>
summary:
This paper introduces a novel Canonical Signed Spike coding for Spiking Neural Networks to enhance performance and reduce latency in ANN-SNN conversions. The CSS incorporates non-linearity by weighting spikes and negative spikes, allowing more information per time step. The authors also propose a Ternary Self-Amplifying neuron model to further reduce latency. Experimental results on CIFAR-10 and ImageNet show that CSS improves inference latency and accuracy with minimal conversion loss, advancing SNN performance on large-scale datasets.

strengths:
The paper's approach of using negative spikes for information encoding is innovative. This technique may compensate for the over firing issue and reduce conversion error.

The one-step silent period seems to be an effective approach which significantly enhances the performance of SNN models.

weaknesses:
The ablation study provided in this paper is insufficient. As it stands, the empirical results primarily support the effectiveness of the negative spike and silent period techniques. A more detailed evaluation of the amplification factor's effect is necessary. Moreover, a comprehensive comparison between the TSA neuron and the standard IF neuron would better highlight the advantages of the proposed neuron model. 

The paper is hard to follow and sometimes confusing. For example, the final neuron function for TSA neurons is not provided in the manuscript, which makes it difficult to fully understand the general behavior of the TSA neuron. Clear and detailed descriptions of such core elements are essential for readability and comprehension.

questions:
Equation 3 makes no sense since the dirac function is a generalized function and is only well-defined within an integral over a specified range. The authors should consider using heaviside step function to describe the neuron firing.

Equation 3 appears to represent the discrete form of a LIF neuron function. However, based on the experiments, the scaling factor $\beta$ is consistently set to values greater than one, which is atypical for a traditional LIF neuron model. 

To improve clarity, it would be beneficial if the authors provide the explicit mathematical function for the TSA neuron model. 

It confuses me whether CSS coding serves as an input encoding method for the SNN or if it refers to the encoding capabilities inherent in the TSA neuron itself.

In lines 305-306, it is mentioned that a new term is added to the right side of Equation 8 after a silent period. Could the authors clarify the purpose of doing so?

It would be better if the authors can provide the pseudo code for the conversion and the inference process.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730558533497

thread_type:
official_review

<thread_object>
creator: reviewer_qunb

<content>
summary:
This paper introduces a novel approach that reduces coding time steps by weighting individual spikes, allowing each spike to transmit increased information density. The introduction of negative spikes successfully breaks temporal dependencies, thereby reducing inference delay. The proposed Compressed Spike Sequence (CSS) encoding method improves both throughput and processing speed in converted Spiking Neural Networks (SNNs), while maintaining minimal conversion accuracy loss, providing an effective solution for enhancing the practical efficiency of SNNs.

strengths:
1.This paper proposes a novel non-linear coding approach named Compressed Spike Sequence (CSS) that breaks the linear relationship constraints inherent in traditional coding schemes.
2.The paper introduces negative spikes and a one-step silent period mechanism that balances coding efficiency and inference latency.
3.The method is comprehensively validated on standard datasets including CIFAR-10 and ImageNet, with extensive testing across network architectures of varying depths (such as VGG and ResNet), and detailed ablation studies analyzing the contribution of each component.
4.The proposed approach achieves nearly lossless ANN-to-SNN conversion with fewer time steps, and can be directly applied to full-precision ANNs without additional quantization training.

weaknesses:
1.The proposed coding scheme lacks sparsity characteristics, which may lead to higher energy consumption.
2.The selection of parameters (such as \beta and \theta) lacks theoretical guidance.
3.Hardware implementation requires processing exponential operations of \beta, handling both positive and negative spikes, and controlling the silent period.

questions:
1.What is the energy consumption of this SNN compared to ANN? It is suggested to add comparative experiments on energy consumption.
2.What is the level of support for this type of neuron in existing neuromorphic hardware? Is deployment and implementation feasible in the future?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732800769281

thread_type:
official_comment

<thread_object>
creator: authors

<content>
comment:
## Further Revisions to Manuscript

__Dear Reviewers__,  

Thank you for your timely follow-up and thoughtful feedback over the past few days. We sincerely appreciate the time and effort you have invested in reviewing our manuscript.

In response to your valuable suggestions, we have further made several revisions to the paper, summarized as follows:

### **Enhancements for Readability**
1. **Additional Figures**: We have added two figures to the main text. One illustrates the information content of spikes at different time steps after weighting, and the other visually demonstrates the phenomenon of temporal coupling of weighted spikes.
2. **Optimized Method Presentation**: Building on the previous version, we have further refined the presentation of the theorems for clarity. Specifically, we removed _Corollary 1_ and upgraded the original _Lemma 1_ to _Theorem 1_, making the approach in the _Methods_ section more direct. Below is a brief summary for the presentation of the _Methods_ section:
   - **Observation 1** focuses the discussion on controlling the residual membrane potential.
   - **Theorem 1** outlines all the factors affecting the residual membrane potential, guiding the introduction of negative spikes and TSA neurons.  
   - With the encoding scheme and neurons clearly defined, **Theorems 2 and 3** establish the connection between CSS-SNN and ANN.

### **Additional Experimental Content**
In response to the feedback during the discussion, we have added the following experiments to the manuscript:
1. The impact of $\beta$ on network performance
2. A comparison with SOTA methods (Fast-SNN, Offset)
3. Energy analysis

Due to space limitations, we _have moved the ablation study on the silent period to the appendix_. However, given the significant role of negative spikes in CSS coding, the ablation study on negative spikes remains in the main text.

---

Overall, these updates integrate the insights from our discussions and have, we believe, significantly improve the quality of the manuscript. We kindly invite you to focus on the sections that interest you most.

Finally, should there be any remaining concerns or questions, please don’t hesitate to reach out. We are happy to address any further issues or provide clarification as needed.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper combines Canonic Signed Spike (CSS) coding scheme with a Ternary Self-Amplifying (TSA) neuron model to mitigate the relatively high time latency problem of ANN-SNN Conversion. Reviewers acknowledge the authors' detailed analysis about CSS coding, but express concerns about the hardware implementation and energy consumption of the converted SNNs in the inference stage. After the discussion period, three reviewers rate 6 and one reviewer rates 3. None of the reviewers strongly argue for acceptance. Therefore, the final decision is not to accept the paper.