Title: Stepwise Weighted Spike Coding for Deep Spiking Neural Networks
Venue: NeurIPS 2024
Accepted: no
Keywords: Spiking neural network

Spike coding scheme

Stepwise weighted spike
Abstract: Spiking Neural Networks (SNNs) seek to mimic the spiking behavior of biological neurons and are expected to play a key role in the advancement of neural computing and artificial intelligence. The efficiency of SNNs is often determined by the neural coding schemes. Existing coding schemes either cause huge delays and energy consumption or necessitate intricate neuron models and training techniques. To address these issues, we propose a novel Stepwise Weighted Spike (SWS) coding scheme to enhance the encoding of information in spikes. This approach compresses the spikes by weighting the significance of the spike in each step of neural computation, achieving high performance and low energy consumption. A Ternary Self-Amplifying (TSA) neuron model with a silent period is proposed for supporting SWS-based computing, aimed at minimizing the residual error resulting from stepwise weighting in neural computation. Our experimental results show that the SWS coding scheme outperforms the existing neural coding schemes in very deep SNNs, and significantly reduces operations and latency.

1

Abstract

Spiking Neural Networks (SNNs) seek to mimic the spiking behavior of biological neurons and are expected to play a key role in the advancement of neural computing and artificial intelligence. The efficiency of SNNs is often determined by the neural coding schemes. Existing coding schemes either cause huge delays and energy consumption or necessitate intricate neuron models and training techniques. To address these issues, we propose a novel Stepwise Weighted Spike (SWS) coding scheme to enhance the encoding of information in spikes. This approach compresses the spikes by weighting the significance of the spike in each step of neural computation, achieving high performance and low energy consumption. A Ternary Self-Amplifying (TSA) neuron model with a silent period is proposed for supporting SWS-based computing, aimed at minimizing the residual error resulting from stepwise weighting in neural computation. Our experimental results show that the SWS coding scheme outperforms the existing neural coding schemes in very deep SNNs, and significantly reduces operations and latency.

1 Introduction

Spiking Neural Networks (SNNs) are known as the third generation of neural network models 16

inspired by the biological structures and functions in the brain [32]. Unlike traditional Artificial 17

Neural Networks (ANNs) that use continuous activation functions, SNNs incorporate discrete spiking 18

events, enabling them to capture temporal dynamics and process information in a manner that closely 19

mimics the brain's functioning [31]. This event-driven paradigm aligns with the brain's energy20

efficient computation and has the potential for more efficient and lower-power computing systems. 21 [33]. 22

Various coding schemes have been proposed to describe neural activities, including rate coding and 23 temporal coding [9]. Rate coding counts the number of spikes fired within a broad time window 24

25

[23, 3, 18, 6], which effectively mitigates the impact of short-term interference on the signal. It was

26

27

widely accepted in the early days and typically outperformed temporal coding [11, 34, 4, 29, 20].

However, the rate coding scheme disregards the information in the temporal domain of the input

spike sequence and requires many pulses to represent the input signal value, making it an inefficient 28

coding method that negates the low-power benefits of SNN. Due to the functional similarity to the 29

biological neural network, spiking neural networks can embrace the sparsity found in biology and 30

are highly compatible with temporal coding [31, 33, 27, 28, 21, 15]. Temporal coding relies on 31

the specific timing or patterns of input spikes, allowing for greater information capacity in a single 32

pulse. However, it requires a large number of time steps to provide fine-grained timing, which 33

increases inference latency. Its sensitivity to variations in spike timing also makes it more vulnerable 34

to temporal jitter or delays [25, 24]. Additionally, decoding temporal-coded information usually 35

requires more complex neuron models [30, 36] and training methodologies [17, 26]. 36

In the study of the temporal information dynamics of spikes, Kim et al. [16] discovered a phenomenon 37 of temporal information concentration in SNNs. It is found that after training, information becomes 38 highly concentrated in the first few timesteps. Based on this observation, we hypothesize that, from 39 the perspective of the postsynaptic neuron, the first arriving spikes contain more information and 40 require stronger responses. Consequently, we propose a mechanism whereby the neuron augments 41 its own membrane potential with a specific coefficient prior to processing the subsequent input. 42 This enhancement serves to increase the importance of preceding pulses on neurons, which is why 43 the spikes are designated as Stepwise Weighted Spikes (SWS). Nevertheless, the amplification of 44 the membrane potential makes it difficult for neurons to reduce its value through traditional "soft 45 reset" (i.e. subtracted by an amount equal to the firing threshold), which can result in residual errors 46 after neuron firing. To address this issue, we make the membrane potential reduced by a magnitude 47 exceeding the threshold after firing. As a result, the membrane potential has both positive and negative 48 residual values, which will generate both positive and negative spikes. This neuron is designated as a 49 Ternary Self-Amplifying (TSA) neuron. To further reduce the error caused by the weighting process, 50 a silent period is incorporated into the TSA neuron, allowing it to receive more input information 51 before firing. We perform the classification tasks with SWS-based SNN on MNIST, CIFAR10, and 52 ImageNet. The results show that the SWS coding scheme can achieve better performance with much 53 fewer coding and computing steps. Even in very deep SNN, SWS coding scheme still performs well 54 and achieves similar accuracy to the ANN with the same structure. Our major contributions to this 55 paper can be summarized as follows: 56

57

58

59

60

61

We propose the SWS coding scheme, which enables easy implementation of SNNs with low energy consumption and high accuracy. The stepwise weighting process enhances the information-carrying capacity of the preceding pulses, greatly reducing the number of coding spikes. Negative pulses are introduced in SWS coding to ensure an accurate information transmission.

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

A novel TSA neuron model is proposed. TSA neuron progressively weights the input by augmenting its residual membrane potential before receiving the subsequent spike. The introduction of negative residual membrane potential and negative thresholds enhances the accuracy of the model's output.

A silent period is added to TSA neuron to markedly improve accuracy at minimal latency cost. By adjusting the silent period step and coding step, SWS-based SNNs can exhibit performance advantages in different aspects, improving the flexibility of applications.

2 Related work

SNNs use spike sequences to convey information, making the encoding of real data into pulses a crucial step. Currently, the mainstream schemes of neural coding are rate coding and temporal coding [9, 33, 32]. Rate coding represents different activities with the number of spikes emitted within a specific time window. Due to its simplicity, rate coding is commonly used in deep learning of SNNs. However, it distributes information uniformly across a large number of spikes, resulting in an inefficient transmission process that increases network latency and energy consumption. Numerous researchers have proposed solutions to optimize inference latency in rate coding. Han et al.[11] proposed a "soft reset" spiking neuron model that retains a residual membrane potential after firing to better mimic the ReLU functionality. They demonstrated near lossless ANN-SNN conversion by using 2-8 times fewer inference time steps. Still, a delay of thousands of steps is required in large datasets or deep networks. In [14], Hu et al. reduced the encode time steps by converting a quantized low-precision ANN to a rate-coded SNN. They also proposed a layer-wise fine-tuning mechanism to minimize the inference latency. However, their neuron model and the subsequent fine-tuning algorithm are relatively complex. Furthermore, in deeper neural networks such as ResNet56, a 1.5% drop in accuracy can be observed. The above rate encoding solutions are limited because they do not consider the significance of each spike.

In [15], Kim et al. proposed phase coding, which assigns different weights to spikes based on their time phase. However, the transmission amount of information is bounded by the global phase, which causes inefficiency in hidden layers, resulting in a latency of up to three thousand steps for a 32-layer network. Burst coding [21] attempts to overcome this issue by introducing burst spikes, which utilize Inter-Spike Interval (ISI). Burst spikes are capable of conveying more information quickly and accurately by inducing Post-Synaptic Potential (PSP) dramatically. Nevertheless, it is still deficient

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

in terms of latency and efficiency. Rueckauer and Liu [27] proposed an efficient temporal encoding scheme where the analog activation values of the ANN neurons are represented by the inverse TimeTo-First-Spike (TTFS) in the SNN neurons. Their new spiking network model generates 7-10 times fewer pulses by utilizing temporal information carried by a single spike. However, as pointed out in [10], TTFS coding scheme incurs expensive memory access and computational overhead, which diminishes the benefit of reduced pulse count. Furthermore, TTFS necessitates a large number of time steps to differentiate between various time points, which also increases network latency. Han and Roy [10] proposed the Temporal-Switch-Coding (TSC) scheme, in which each input image pixel is represented by two spikes, and its intensity is proportional to the timing between the two pulses. Their results showed a reduction in energy expenditure. However, TSC coding requires a large number of time steps to provide distinguishable time intervals, rendering it an ineffective approach to addressing the issue of the long latency.

Overall, rate coding employs a large number of pulses to encode information, which results in a considerable energy overhead and inference delays. On the other hand, temporal coding allows for greater information capacity in a single spike, but this does not reduce the computing latency as a precise time point or period can be identified only with a sufficient number of time steps. Therefore, new neural coding schemes should be developed.

3 Stepwise weighted spike coding scheme

3.1 Stepwise weighting

The spike train S l i ( t ) of the i th neuron in the l th layer can be expressed as follows:

where δ ( t ) is the Dirac delta function, θ l is the spike amplitude of the l th layer, which is usually set to the same value as the firing threshold. f is the index of the spike in the sequence, and F l i denotes a set of spike times which satisfies the firing condition:

where u l i ( t ) denotes the membrane potential and V l th denotes the firing threshold of the neurons in the l th layer.

Our basic idea is to amplify the membrane potential before the receipt of the subsequent input, which amplifies and prolongs the impact of the preceding input spikes on membrane potential, emulating the phenomenon of information concentration identified in [16]. For clarity, the meanings of important symbols are provided in table 1. The action of a neuron in SWS-SNN can be described as follows:

where β is the amplification factor which should be greater than one, z l j ( t ) denotes the PSP (i.e. integrated inputs):

where ω ij is the synaptic weight and b l j is the bias. Begin with the initial value u l j (0) = 0 and iteratively apply eq. (3) for each subsequent value until u l j ( n ) and substitute eq. (1) and eq. (4) into it,

eq. (3) can be written as: 125

126

127

128

129

130

131

132

133

134

135

136

137

138

Note that S l j ( t ) is set to zero for simplicity. From eq. (5), it can be seen that the stepwise augment of the membrane potential results in the spike input at time t l -1 , ( f ) i encoding the value θ l -1 β n -t l -1 , ( f ) i . This process is thus referred to as stepwise weighting, and β n -t l -1 , ( f ) i serves as the weight. The earlier the input pulse, the greater its ability to carry information. This solves the problem of excessive encoding steps in previous schemes, allowing faster information transmission.

3.2 Residual error

Stepwise weighting effectively assigns more weight to earlier arriving pulses, but it also makes spike generation more tricky. To ensure that input information is efficiently encoded and transmitted to the next layer, the residual membrane potential should be minimized after neural computation is completed. The stepwise weighting, however, amplifies the residual potential from the previous time step. If z l j ( t ) remains high in subsequent steps, reducing the membrane potential becomes challenging, as shown in fig. 1(b). This vicious cycle ultimately leads to a persistently high membrane potential, indicating that a substantial amount of information remains unencoded.

We refer to this phenomenon as residual error. One contributing factor is that the threshold is set 139 too high, resulting in a pulse being emitted only when the membrane potential exceeds the value θ l . 140 While this prevents excessive information transmission, it results in missed opportunities to bring 141 down u l j ( t ) by firing a spike. 142

To address this issue, we propose setting the firing threshold V l th to 1 2 θ l . This adjustment facilitates 143 pulse generation and reduces the residual membrane potential. After the neuron firing, the membrane 144 potential is subtracted by θ l , which leads to the emergence of a negative residual that will be stepwise 145

weighted over time. The coefficient 1 /2 is selected as it is capable of controlling both positive and 146 negative residuals within a narrow and balanced range. A negative threshold -V l th is introduced into 147 the neuron model, which initiates a negative spike when the membrane potential falls below this 148 threshold. This mechanism allows the excessively emitted information to be corrected by the negative 149 spike, as shown in fig. 1(d). Given the above characteristics, we designate this neuron model as a 150 TSA neuron. 151

152

153

154

155

156

157

158

159

160

161

162

3.3 Silent period

Another contributing factor to residual error is the imbalanced distribution of z l j ( t ) . A burst input of z l j ( t ) at time point τ results in a sharp rise in membrane potential, making it difficult for subsequent spikes to reduce it, as shown in fig. 2(a).

This can be addressed by incorporating a silent period T s into the TSA neuron model. The neurons only integrates input and performs stepwise weighting, but are not allowed to fire in the first T s steps. This enables the acquisition of more known information before spike generation, resulting in increased accuracy, as illustrated in fig. 2(b). Since the preceding input information has been amplified by β T s after the silent period, V l th also needs to be adjusted accordingly, which is set to β Ts 2 θ l . Similarly, after firing, the membrane potential should be subtracted by θ l β T s . Note that the fired spike amplitude remains unchanged, that is, θ l .

The impact of the silent period on network latency is shown in fig. 2(d). The output results for 163 different input sequences are distinguished by blocks of different colors. It can be observed that 164 as network depth increases, the silent period accumulates, leading to a higher output latency. The 165 inference latency of SWS-SNN can be calculated as follows: 166

where T inf is the inference delay, T c is the coding time steps, T s is the length of the silent period and 167 L TSA is the number of TSA neuron layers. The neuron model in other coding schemes yields a zero 168 T s , leading to an output delay equal to the coding time step, which is consistent with the definition in 169 the previous scheme. From fig. 2(d), it can be seen that different input sequences are processed in a 170 pipeline-like manner, and the value of T c + T s determines the throughput rate of SWS-SNN. 171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

3.4 Input encoding

According to eq. (5), the value that can be losslessly encoded under the SWS coding scheme can be expressed as follows:

where A j denotes the encoded value. a τ j ∈ {-1 , 0 , 1 } indicates the type of the output spike at time τ : 1 for a positive pulse, -1 for a negative pulse and 0 for no pulse. T c denoted the time steps used for encoding. The weight β T c -τ results from the stepwise weighting process described in section 3.1. θ 0 denotes the spike amplitude of the input encoding layer, which can be assigned an appropriate value based on the range to be encoded.

According to eq. (7), given a fixed T c and θ 0 , the distribution of A j is determined by β . Setting β to 2 is reasonable, as it ensures A j is evenly distributed within the codable range. Compared to rate coding, which necessitates 2 T c coding steps to encode the same range with same precision, SWS coding significantly enhances coding efficiency. Note that with the introduction of negative pulses, setting β to 3 can also achieve a uniform distribution of A j and offers even more values for accurate encoding compared to β = 2 . 1 When β is less than 2 , the distribution of A j becomes denser at smaller values, which may be suitable for encoding data that follows a similar distribution.

For static image classification tasks, the pixel value p j can be encoded by applying a constant input z 0 j ( t ) to the TSA neuron. Considering the stepwise weighting process, we can write:

where ∣ ∣ z 0 j ∣ ∣ denotes the amplitude of the constant input z 0 j ( t ) . Solve for ∣ ∣ z 0 j ∣ ∣ and we have: 189

Given that z 0 j ( t ) is a constant at each step, T s can be set to 0 for the encoding layer. However, the 190 neuron must await T s time steps after the completion of an encoding. This allows neurons in the 191 subsequent layer to complete the previous neural computing before receiving the next encoded input. 192

193

194

195

196

197

198

199

200

201

4 Experiments

In this section, we convert quantized ANNs to SWS-based SNNs 2 and conduct experiments on MNIST, CIFAR10, and ImageNet. Firstly, an overview of SWS-SNN's performance across various datasets is provided. Subsequently, the network's inference latency and energy consumption is compared with other spike coding schemes. Finally, an ablation study is conducted to investigate the impact of lowered thresholds and silent periods on reducing residuals and enhancing accuracy.

ANNs used for conversion are all quantized to 8 bits. β is set to 2 in the experiments to ensure that codable values are evenly distributed. Compared to β = 3 , a smaller amplification factor reduces the impact of residual errors, resulting in more accurate output.

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

4.1 Overall performance

For simple classification tasks such as CIFAR10, our proposed SWS coding scheme has a faster inference speed than other ANN-SNN models while achieving similar classification accuracy, or has higher classification accuracy than direct learning at similar inference speeds. For example, ResNet18 with SWS improves throughput seven times over [20] while simultaneously improving accuracy. Although the network in [5] has a slightly higher throughput, its accuracy is 1.17% lower than our scheme. To fully test the potential of our proposed coding scheme, we conducted experiments on ImageNet using networks with various structures. The experimental results demonstrate that SWS coding has distinct advantages on extremely deep SNNs. Our SWS-based ResNet50 and ResNeXt101 achieved over 80% accuracy on ImageNet with only eight coding steps. The model in [12] achieves an almost lossless conversion with eight time steps. However, their method has to adjust the resting potential of neurons layer by layer, and the calibration effect for deeper networks is unclear. In [14], the original ANN needs to be quantized to 3 bits, resulting in a larger conversion loss. Directly trained SNNs typically achieve higher throughput, but their accuracy still requires improvement. In addition, the SWS coding scheme is easy to implement. No further fine-tuning is required after the conversion.

4.2 Accuracy vs. latency

The comparison of latency results between SWS-SNN and other ANN-converted SNNs[1, 11, 10, 4, 18, 2, 7] is illustrated in fig. 3. The latency of the network is calculated with eq. (6). In the counterpart models, the variation of delay is mainly caused by the changes in T c . In contrast, T s determines latency in deep SWS-SNNs. Therefore, SWS-SNN has an upper limit on latency: T max inf = T c (1 + L TSA ) , which causes our curve to terminate earlier in fig. 3.

To ensure a fair comparison, we represent the ANN accuracy of each counterpart with dotted lines of the same color. The experimental results indicate that SWS-SNN can achieve optimal performance with minimal latency. Specifically, SWS-based VGG-16 can converge to the ANN performance

226

227

228

229

230

231

232

233

2 4

2 5

2 6

2 7

2 8

2 9

Latency

88

89

90

91

92

93

94

95

96

CIFAR10 acc (%)



TWS (ours)

RMP

Calibration

OPI

Opt

RNL

2 4

2 5

2 6

2 7

2 8

2 9

2 10

Latency

56

58

60

62

64

66

68

70

72

74

76

ImageNet acc (%)



TWS (ours)

RMP

Calibration

OPI

QCFS

TSC

2 4

2 5

2 6

2 7

2 8

2 9

2 10

Latency

54

56

58

60

62

64

66

68

70

72

74

76

ImageNet acc (%)



TWS (ours)

RMP

Calibration

QCFS

TSC

















OPF (M)

















MNIST acc (%)



Ts=0

Ts=1

Ts=2

















OPF (M)



















MNIST acc (%)



SWS (Tc=8)

SWS (Tc=6)

SWS (Tc=5)

SWS (Tc=4)

Rate (44 steps)

Rate (18 steps)

TTFS (base)

TTFS (dyn thresh)

TTFS (clamped)

Pattern (4bits)

Pattern (8bits)

TC=4

Tc=5

Tc=6

Tc=7

Tc=8

in the shortest time on CIFAR10 and reduce the inference latency on ImageNet by more than one order. Even though the silent period accumulates when the network gets deeper, the results in fig. 3(c) demonstrate that our scheme still achieves the fastest inference speed with the highest accuracy in a 34 -layer network. Note that T s is set to the same value for each TSA layer for simplicity, resulting in discontinuous T inf values. This causes a sharp drop in accuracy at smaller delays.

4.3 Operation counting

To compare the energy consumption of SWS-SNN with SNNs under other encoding schemes, we adopt the method as in [29, 27, 28] to count operations:

where OPF (Operations Per Frame) denotes the number of operations for the classification of one 234 frame, T c and T s denotes the coding steps and the length of the silent period, respectively. L TSA 235 denotes the number of TSA layers, f l out denotes the fan-out of neurons in layer l , n l ( t ) denotes the 236 number of spikes fired in layer l at time τ and N TSA denotes the number of TSA neurons. The 237 first term on the right-hand side of the equation arises from the TSA's requirement to amplify the 238 membrane potential. Note that due to the accumulation of T s over the network depth, the time period 239 for counting n l ( t ) varies with l . 240

Experiments were conducted on MNIST using LeNet-5. We varied the silent periods and adjusted 241 the coding steps to study their effects on OPF. The results are presented in fig. 4(a). As indicated in 242 eq. (10), reducing T c lowers energy overhead. This presents a trade-off between energy consumption 243 and inference accuracy, as fewer coding steps also reduce the number of values that can be accurately 244 encoded. A larger T s requires TSA neurons to perform more operations to amplify the membrane 245 potential. On the other hand, it reduces the number of unnecessary pulse emissions. Overall, silent 246 period has a negligible impact on OPF. 247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6

Residual (normalized)

0

20

40

60

80

100

Prob. density



V l

th

=

l

,

Ts

=0

V l

th

=

l

,

Ts

=1

V l

th

=

l

,

Ts

=2

V l

th

=

l

2

,

Ts

=0

V l

th

=

l

2

,

Ts

=1

V l

th

=

l

2

,

Ts

=2

0.20

0.25

0.30

200

250

300

350

400

0

20

40

60

80

100

CIFAR10 acc (%)

9.91

84.21

94.89

35.41

95.67

95.68



V l

th

=

l

,

Ts

=0

V l

th

=

l

,

Ts

=1

V l

th

=

l

,

Ts

=2

V l

th

=

l

2

,

Ts

=0

V l

th

=

l

2

,

Ts

=1

V l

th

=

l

2

,

Ts

=2

In fig. 4(b), the energy consumption of SWS-based SNN is compared with that of other SNNs. The experimental results demonstrate that our coding scheme can achieve a favorable balance between accuracy and energy consumption. The SWS coding scheme is superior to rate coding and temporal pattern coding in that it requires fewer operations and achieves higher accuracy. In TTFS encoding, each neuron fires at most one spike at a time, theoretically demanding the least OPF. With T c = 4 , SWS-SNN can achieve significantly higher accuracy with minimal increase in OPF. Note that if the ANN is quantized to a lower number of bits (e.g., 4 bits), the error caused by the reduced T c can actually be compensated by the quantization algorithm, which can potentially result in a higher performance.

4.4 Ablation study

In section 3.2 and section 3.3, we proposed reducing the firing threshold and introducing a silent period to mitigate residual error. To assess the impact of these two adjustments, we conducted experiments on CIFAR10 using ResNet18. After the neural computation, the residuals (absolute values) of the TSA neurons were analyzed. We first scaled the residuals by 1 / β Ts to counteract the effect of membrane potential amplification caused by the silent period, and then normalized them in units of θ l . The probability density of the residuals is shown in fig. 5(a).

The results demonstrate that lowering V l th shifts the residual distribution from around 0 . 5 θ l to approximately 0 . 25 θ l , corresponding to the quantization errors (i.e. rounding errors) under their respective thresholds. The addition of silent periods further concentrates the distribution and reduces large deviations. As can be seen from the green curve in fig. 5(a), setting T s to 2 and V l th to θ l / 2 makes the residuals almost all distributed around the quantization error. Compared to the red curve (without a lowered V l th or a silent period), the residuals are greatly reduced, which fully proves the effectiveness of lowering the threshold and adding a silent period. The inference results on CIFAR10 is shown in fig. 5(b). When setting V l th to θ l and T s to zero, the network's output is almost random. Lowering the threshold and adding a silent period improve the accuracy to 35 . 41% and 84 . 21% , respectively. Ultimately, the combination of both adjustments enabled SWS-ResNet18 to achieve an accuracy of 95 . 68% on CIFAR10.

5 Conclusion

In this work, we have proposed a novel SWS spike coding scheme. The stepwise weighting process 276 enhances the information-carrying capacity of the preceding pulses, greatly reducing the number of 277 time steps for encoding. Combined with a silent period, our proposed TSA neuron model solves the 278 problem of residual errors and achieves fast and accurate information transmission. Our experimental 279 results have demonstrated that SWS coding is highly effective in extremely deep SNNs and achieves 280 state-of-the-art accuracy. The SWS coding scheme is also highly flexible and can adapt to various 281 needs. 282

References 283

Bu, T., Ding, J., Yu, Z., Huang, T.: Optimized potential initialization for low-latency spiking 284 neural networks (2022) 285

Bu, T., Fang, W., Ding, J., Dai, P., Yu, Z., Huang, T.: Optimal ann-snn conversion for high286 accuracy and ultra-low-latency spiking neural networks (2023) 287

Cao, Y., Chen, Y., Khosla, D.: Spiking deep convolutional neural networks for 288 energy-efficient object recognition. International Journal of Computer Vision 113 (1), 54289 66 (May 2015). https://doi.org/10.1007/s11263-014-0788-3, https://doi.org/10.1007/ 290 s11263-014-0788-3 291

Deng, S., Gu, S.: Optimal conversion of conventional artificial neural networks to spiking neural 292 networks (2021) 293

Deng, S., Li, Y., Zhang, S., Gu, S.: Temporal efficient training of spiking neural network via 294 gradient re-weighting (2022) 295

Diehl, P.U., Neil, D., Binas, J., Cook, M., Liu, S.C., Pfeiffer, M.: Fast-classifying, 296 high-accuracy spiking deep networks through weight and threshold balancing. In: 297 2015 International Joint Conference on Neural Networks (IJCNN). pp. 1-8 (2015). 298 https://doi.org/10.1109/IJCNN.2015.7280696 299

Ding, J., Yu, Z., Tian, Y ., Huang, T.: Optimal ann-snn conversion for fast and accurate inference 300 in deep spiking neural networks (2021) 301

Fang, W., Yu, Z., Chen, Y., Huang, T., Masquelier, T., Tian, Y.: Deep residual learning in 302 spiking neural networks (2022) 303

Guo, W., Fouda, M.E., Eltawil, A.M., Salama, K.N.: Neural coding in spiking neural net304 works: A comparative study for robust neuromorphic systems. Frontiers in Neuroscience 305 15 (2021). https://doi.org/10.3389/fnins.2021.638474, https://www.frontiersin.org/ 306 journals/neuroscience/articles/10.3389/fnins.2021.638474 307

Han, B., Roy, K.: Deep spiking neural network: Energy efficiency through time based coding. 308 In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.M. (eds.) Computer Vision - ECCV 2020. pp. 309 388-404. Springer International Publishing, Cham (2020) 310

Han, B., Srinivasan, G., Roy, K.: Rmp-snn: Residual membrane potential neuron for enabling 311 deeper high-accuracy and low-latency spiking neural network (2020) 312

Hao, Z., Ding, J., Bu, T., Huang, T., Yu, Z.: Bridging the gap between anns and snns by 313 calibrating offset spikes (2023) 314

Hu, Y., Tang, H., Pan, G.: Spiking deep residual networks. IEEE Transac315 tions on Neural Networks and Learning Systems 34 (8), 5200-5205 (2023). 316 https://doi.org/10.1109/TNNLS.2021.3119238 317

Hu, Y., Zheng, Q., Jiang, X., Pan, G.: Fast-snn: Fast spiking neural network by converting 318 quantized ann. IEEE Transactions on Pattern Analysis and Machine Intelligence 45 (12), 14546319 14562 (2023). https://doi.org/10.1109/TPAMI.2023.3275769 320

Kim, J., Kim, H., Huh, S., Lee, J., Choi, K.: Deep neural networks with weighted spikes. Neuro321 computing 311 , 373-386 (2018). https://doi.org/https://doi.org/10.1016/j.neucom.2018.05.087, 322 https://www.sciencedirect.com/science/article/pii/S0925231218306726 323

Kim, Y., Li, Y., Park, H., Venkatesha, Y., Hambitzer, A., Panda, P.: Exploring temporal 324 information dynamics in spiking neural networks (2022) 325

Lee, C., Sarwar, S.S., Panda, P., Srinivasan, G., Roy, K.: Enabling spike-based back326 propagation for training deep neural network architectures. Frontiers in Neuroscience 327 14 (Feb 2020). https://doi.org/10.3389/fnins.2020.00119, http://dx.doi.org/10.3389/ 328 fnins.2020.00119 329

330

331

332

333

334

335

336

337

338

339

340

341

342

343

344

345

346

347

348

349

350

351

352

353

354

355

356

357

358

359

360

361

362

363

364

365

366

367

368

369

370

371

372

373

374

375

376

377

Li, Y., Deng, S., Dong, X., Gong, R., Gu, S.: A free lunch from ann: Towards efficient, accurate spiking neural networks calibration (2021)

Meng, Q., Xiao, M., Yan, S., Wang, Y., Lin, Z., Luo, Z.Q.: Training high-performance lowlatency spiking neural networks by differentiation on spike representation (2023)

Meng, Q., Yan, S., Xiao, M., Wang, Y., Lin, Z., Luo, Z.Q.: Training much deeper spiking neural networks with a small number of time-steps. Neural Networks 153 , 254-268 (2022). https://doi.org/https://doi.org/10.1016/j.neunet.2022.06.001, https://www.sciencedirect. com/science/article/pii/S0893608022002064

Park, S., Kim, S., Choe, H., Yoon, S.: Fast and efficient information transmission with burst spikes in deep spiking neural networks (2019)

Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Köpf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., Chintala, S.: Pytorch: An imperative style, high-performance deep learning library (2019)

Pérez-Carrasco, J.A., Zhao, B., Serrano, C., Acha, B., Serrano-Gotarredona, T., Chen, S., Linares-Barranco, B.: Mapping from frame-driven to frame-free event-driven vision systems by low-rate rate coding and coincidence processing-application to feedforward convnets. IEEE Transactions on Pattern Analysis and Machine Intelligence 35 (11), 2706-2719 (2013). https://doi.org/10.1109/TPAMI.2013.71

Querlioz, D., Bichler, O., Dollfus, P., Gamrat, C.: Immunity to device variations in a spiking neural network with memristive nanodevices. IEEE Transactions on Nanotechnology 12 (3), 288-295 (2013). https://doi.org/10.1109/TNANO.2013.2250995

Querlioz, D., Bichler, O., Gamrat, C.: Simulation of a memristor-based spiking neural network immune to device variations. In: The 2011 International Joint Conference on Neural Networks. pp. 1775-1781 (2011). https://doi.org/10.1109/IJCNN.2011.6033439

Rathi, N., Srinivasan, G., Panda, P., Roy, K.: Enabling deep spiking neural networks with hybrid conversion and spike timing dependent backpropagation (2020)

Rueckauer, B., Liu, S.C.: Conversion of analog to spiking neural networks using sparse temporal coding. In: 2018 IEEE International Symposium on Circuits and Systems (ISCAS). pp. 1-5 (2018). https://doi.org/10.1109/ISCAS.2018.8351295

Rueckauer, B., Liu, S.C.: Temporal pattern coding in deep spiking neural networks. In: 2021 International Joint Conference on Neural Networks (IJCNN). pp. 1-8 (2021). https://doi.org/10.1109/IJCNN52387.2021.9533837

Rueckauer, B., Lungu, I.A., Hu, Y., Pfeiffer, M., Liu, S.C.: Conversion of continuous-valued deep networks to efficient event-driven networks for image classification. Frontiers in Neuroscience 11 (2017). https://doi.org/10.3389/fnins.2017.00682

Stöckl, C., Maass, W.: Optimized spiking neurons can classify images with high accuracy through temporal coding with two spikes. Nature Machine Intelligence 3 (3), 230238 (Mar 2021). https://doi.org/10.1038/s42256-021-00311-4, https://doi.org/10.1038/ s42256-021-00311-4

Taherkhani, A., Belatreche, A., Li, Y., Cosma, G., Maguire, L.P., McGinnity, T.: A review of learning in biologically plausible spiking neural networks. Neural Networks 122 , 253-272 (2020). https://doi.org/https://doi.org/10.1016/j.neunet.2019.09.036, https://www. sciencedirect.com/science/article/pii/S0893608019303181

Wang, X., Lin, X., Dang, X.: Supervised learning in spiking neural networks: A review of algorithms and evaluations. Neural Networks 125 , 258-280 (2020). https://doi.org/https://doi.org/10.1016/j.neunet.2020.02.011, https://www.sciencedirect. com/science/article/pii/S0893608020300563

378

379

380

381

382

383

384

385

386

387

388

389

390

391

392

393

394

395

396

397

Yamazaki, K., Vo-Ho, V.K., Bulsara, D., Le, N.: Spiking neural networks and their applications: A review. Brain Sci 12 (7) (Jun 2022)

Yan, Z., Zhou, J., Wong, W.: Near lossless transfer learning for spiking neural networks. In: AAAI Conference on Artificial Intelligence (2021), https://api.semanticscholar.org/ CorpusID:235349069

Zheng, H., Wu, Y., Deng, L., Hu, Y., Li, G.: Going deeper with directly-trained larger spiking neural networks (2020)

Zhou, S., LI, X., Chen, Y., Chandrasekaran, S.T., Sanyal, A.: Temporal-coded deep spiking neural network with easy training and robust performance (2021)

A Appendix

A.1 Convert quantized ANNs to SWS-SNNs

A pretrained ANN was first obtained from torchvision, which is part of the PyTorch[22] project, and then quantized into n bits following the Quantization-Aware Training (QAT) Workflow provided by PyTorch ( 8 bits in the actual experiment, with n bits used here for generality). The quantized ANN can be characterized by the parameters listed in table 3, and the basic idea of the conversion process is illustrated in fig. 6(a). The activations of the quantized ANN can be mapped to an integer Q between [0 , 2 n -1] using a scaling factor C and a zero point Z . With the same weight and bias between Q l i and Q l o , the TSA layer can generate S l , which encodes Q l o , provided that S l -1 encodes Q l i and no residual error occurs. In the actual SNN, the pulse amplitude θ l is normalized to 1 . Therefore, the bias need to be further scaled to derive the final weight W l and bias b l for the SWS-SNN.

The derivation is as follows. After QAT, we have: 398

where Q l i , Q l o represent the integers to which the quantized input and output are mapped, respectively. 399 Substitute eq. (12) and eq. (13) into eq. (11), and we can write: 400

which gives: 401

407

408

409

410

411

412

413

414

415

416

417

418

419

420

421

422

423

where 402

As seen in eq. (15), with the weight and bias set to ˜ W l and ˜ b l respectively, the layer outputs Q l o when 403 receiving Q l i . The pulse amplitude θ l can be set to any value as long as the codable range calculated 404 by eq. (7) covers [0 , 2 n -1] . Then we have: 405

Considering the membrane potential amplification, b l can be calculated as follows: 406

Once the T c , β and θ l ( θ l -1 is given by the previous layer) have been determined, all values on the right side of eq. (18) and eq. (19) are known. Consequently, W l and b l in the SWS-SNN can be readily calculated from the weight and bias of the quantized ANN.

After configuring the weights and biases as described above, the input pixel must be encoded into a pulse sequence with an amplitude of 1 as well. This process is illustrated in fig. 6(b). First, map the pixel value to [0 , 2 n -1] using C 0 i and Z 0 i obtained from QAT. Assuming this range can be encoded by SWSs with an amplitude of θ 0 , scaling the pixel value by 1 / θ 0 allows the use of a sequence with θ 0 = 1 for encoding. Finally, encode the scaled pixels following section 3.4, and the required input spike sequence is obtained.

A.2 Details for QAT

QAT is the quantization method that typically results in the highest accuracy. We basically follows the workflow provided by PyTorch. The default QAT quantization configuration is chosen to specify the kind of fake-quantization inserted after weights and activations. We choose Stochastic Gradient Descent (SGD) optimizer in QAT, with the value of momentum set to 0 . 9 and the learning rate set to 1 × 10 -4 since the weights only need to be fine-tuned. QAT is done for 12 epochs and 20 batches in each epoch. We freeze the batch norm mean and variance estimates after three epochs and freeze the quantizer parameters (scaling factor and zero point) after another two epochs.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: Stepwise weighting enhances the encoding of information in spikes, as is proved in eq. (5) in section 3.1. Our proposed SWS coding scheme achieves high performance and low energy consumption, which is supported by our experimental results in section 4. The TSA neuron model effectively minimizes the residual error, which can be proved from the ablation study in section 4.4.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: The inclusion of silent periods can lead to increased latency, as noted in section 3.3, which is a limitation we've found so far. However, our experimental results demonstrate that our delay performance still surpasses that of other SNNs.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: The membrane potential amplification enhances the information-carrying capacity of the preceding pulses and is proved in eq. (5).

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We set specific random number seeds when conducting experiments to ensure that all the results of section 4 are reproducible.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility.

530

531

532

533

534

535

536

537

538

539

540

541

542

543

544

545

546

547

548

549

550

551

552

553

554

555

556

557

558

559

560

561

562

563

564

565

566

567

568

569

570

571

572

573

574

575

576

577

578

579

580

581

In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [No]

Justification: Code will be released when the paper is accepted.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: The details for acquiring different delays and the OPF calculation method are provided in section 4.2 and section 4.3, respectively. The parameters used during QAT training is outlined in appendix A.2.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [No]

Justification: We believe it is not necessary to include error bars in the results because each experimental result itself is already the average of a large number of tests (E.g., the test accuracy for an epoch is averaged over Num _ of _ batches × Batch _ size input images, and is therefore very close to each other in every test epoch).

Guidelines:

582

583

584

585

586

587

588

589

590

591

592

593

594

595

596

597

598

599

600

601

602

603

604

605

606

607

608

609

610

611

612

613

614

615

616

617

618

619

620

621

622

623

624

625

626

627

628

629

630

631

632

633

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [No]

Justification: We found it difficult to quantify the computing resources used in every experiments.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We have read the NeurIPS Code of Ethics and the research conducted in this paper conforms with it.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: There is no societal impact of the work performed.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The paper poses no such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: The pretrained ANN model and the QAT workflow is provided by PyTorch and we cited the original paper in appendix A.1 as [22].

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: The paper does not release new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

738

739

740

741

742

743

744

745

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work belongs to ANN2SNN and proposes a novel coding scheme and neuron model to enhance the efficiency and accuracy of Spiking Neural Networks (SNNs) while reducing energy consumption. The Stepwise Weighted Spike (SWS) coding scheme improves information encoding by stepwise weighting input signals and introducing negative pulses, reducing the number of coding spikes needed. The Ternary Self-Amplifying (TSA) neuron model further enhances accuracy by progressively weighting the input through residual membrane potential adjustments and incorporating negative residuals and thresholds. Introducing silent periods allows the neuron to receive more input information before firing, significantly improving accuracy with minimal latency. Experimental results on datasets like MNIST, CIFAR10, and ImageNet demonstrate that the SWS coding scheme achieves better performance with fewer coding and computing steps, performing well even in very deep SNNs and achieving accuracy comparable to Artificial Neural Networks (ANNs) with the same structure.\n\nSTRENGTHS:\nOriginality: This work introduces the Stepwise Weighted Spike (SWS) coding scheme, which is a novel approach in the field of Spiking Neural Networks (SNNs). The proposed method compresses spikes by weighting their significance in each step of neural computation, which enhances the performance and reduces the energy consumption of SNNs. Ternary pulses are a relatively new method in SNN, so the improvement of the ternary SNN encoding method has a relatively high degree of originality.\n\nQuality: The paper provides a comprehensive set of experiments to validate the proposed SWS coding scheme. These experiments demonstrate that the SWS coding scheme significantly reduces operations and latency compared to existing neural coding schemes. The paper outlines the parameters used during training and provides justifications for the chosen experimental settings. \n\nClarity: The introduction of the paper effectively motivates the work by discussing the limitations of current SNN coding schemes and proposing SWS as a solution. The methodology is clearly presented, with detailed descriptions of the new coding scheme and the Ternary Self-Amplifying (TSA) neuron model. Important symbols and their meanings are well-explained, contributing to the overall clarity of the paper.\n\nSignificance: The paper makes a significant contribution by proposing the SWS coding scheme, which enhances the efficiency and performance of SNNs. This new method addresses critical issues such as high latency and energy consumption in existing coding schemes, making it a valuable addition to the field. By improving the encoding of information in spikes, the SWS scheme has the potential to advance the development of more efficient and lower-power computing systems, thereby providing new options for the choice of coding schemes in SNNs.\n\nWEAKNESSES:\nIn the ImageNet experiments in Table 2, SWS and other comparative ANN-SNN methods used different baselines, which is why the '$SNN\\  Acc$' results are much higher than those of the comparative methods. However, the ‘$\\Delta ACC$’ does not seem to show a significant difference (except for Hybrid training and Spiking ResNet). Using the same network architecture and pre-trained weights would be more credible.\n\nQUESTIONS:\n1)  \n\nFor the experiments on ImageNet in Table 2, it seems that the ANN baseline of SWS has a higher accuracy than that for the comparative methods. Why weren't other methods tested on the same architecture and pre-trained weights?\n\n2)  \n\nFigure 2 describes the 'silent period' proposed in this paper, during which neuron potentials accumulate and are not allowed to fire spikes. After experiencing a silent period of $T_s$, the corresponding $\\theta^l$ and $V_{th}^l$ are amplified by a factor of $\\beta^{T_s}$. Firstly, should the $V_{th}^l$ in Figures 2(b) and 2(c) be $V_{th}^l\\beta^{T_s}$? Otherwise, it does not correspond to $\\theta^l\\beta^{T_s}$. Secondly, how many time steps do the 'burst' spikes last after the silent period? Why, in the middle graph of Figure 2(b), are there larger spikes at the second and fourth time steps and a smaller spike at the third time step? In the middle graph of Figure 2(c), why is there a smaller spike at the third time step and a larger spike at the fourth time step? Is the amplification of $V_{th}^l$ and $\\theta^l$ by $\\beta^{T_s}$ maintained for several time steps or continuously after the silent period?\n\nLIMITATIONS:\nThe authors have not explicitly addressed the limitations or potential negative societal impacts of their work. To improve the transparency and completeness of their research, the authors could consider the following constructive suggestions:\n\n1)  Limitations:\n\nCreate a dedicated \"Limitations\" section in the paper to discuss any constraints, assumptions, or potential weaknesses of the proposed SWS coding scheme.\n\nReflect on the robustness of the results to violations of assumptions, such as noiseless settings, model specifications, or dataset dependencies.\n\nDiscuss the scope of the claims made in the paper, including the generalizability of the approach across different datasets and scenarios.\nAddress factors that may influence the performance of the SWS coding scheme, such as computational efficiency and scalability with varying dataset sizes.\n\nConsider possible limitations related to privacy and fairness concerns in the implementation of the SWS coding scheme.\n\n2)  Negative Societal Impact:\n\nExplicitly acknowledge the potential negative societal impacts of the SWS coding scheme, such as privacy risks, fairness considerations, or unintended consequences.\n\nDiscuss how the technology could be misused or lead to harmful outcomes, even if not intended by the authors.\n\nConsider mitigation strategies to address any identified negative societal impacts, such as controlled release of models, monitoring mechanisms, or additional safeguards.\n\nEmphasize the importance of ethical considerations and responsible deployment of the SWS coding scheme in real-world applications.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel Stepwise Weighted Spike (SWS) coding scheme designed to improve the efficiency of Spiking Neural Networks (SNNs) by compressing spikes and weighting their significance in each step of neural computation. This method addresses the issues of high delays and energy consumption associated with existing SNN coding schemes, as well as the complexity of neuron models and training techniques. The authors also introduce a Ternary Self-Amplifying (TSA) neuron model, incorporating a silent period to support SWS-based computing. This model is designed to minimize the residual error resulting from the stepwise weighting process. The experimental results provided in the manuscript demonstrate that the proposed SWS coding scheme significantly outperforms existing neural coding schemes, particularly in very deep SNNs. Key improvements include reduced operations and latency, enhanced overall performance, and lower energy consumption.\n\nSTRENGTHS:\n1.\tInnovative Approach: Introducing the SWS coding scheme and TSA neuron model is innovative.\n2.\tPerformance Improvement: This paper provides experiments showing that the proposed methods outperform existing coding schemes regarding both performance and energy efficiency.\n\nWEAKNESSES:\n1.\tClarity and Detail: Some sections of this paper could benefit from more detailed explanations, particularly in the description of the SWS coding scheme and TSA neuron model. This would help in understanding the underlying mechanisms and their advantages.\n2.\tComparative Analysis: While the experimental results are promising, there is no proof from the experimental results that the encoding method proposed is more advantageous.\n3.\tThere are some grammatical errors in the paper. Such as the second paragraph of Section 3.3, \"The neurons only integrates input and performs stepwise weighting\". It is recommended that a uniform representation be used for \"spike\" and \"pulse\".\n4.\tSymbol design problem, \"t\" in Eq. (3) becomes \"n\" in Eq. (5).\n5.\tThere are many long paragraphs and sentences in the paper, making it difficult for readers to accurately understand the meaning of the paper.\n6.\tThe description of the problem in the third paragraph of Section 1 and the end of Section 2 is not clear, making it difficult for readers to understand the problem that the article really wants to solve.\n7.\tThe description of the encoding method in Eq. (7) is difficult to understand. According to Eq.  (7), the encoded value $A_j$ should have no time step. However, in the experimental part, the method of this paper has 8 time steps.\n\nQUESTIONS:\n1.\tThe paper proposes a new encoding method. Is this encoding method designed for the TSA neuron model, or is it an encoding method in more general scenarios? If it is a targeted approach, would it be more appropriate as a sub-module of the proposed method?\n2.\tThree data sets are used in the experimental section. Why don’t show the comparison results on MNSIT in Table 2?\n\nLIMITATIONS:\nThe paper mentioned that due to the setting of the neuron's silent period, the delay increases. It can be seen from the experiments that the overall latency of the method is lower, which can be regarded as solving this limitation. At the same time, this article does not have potential negative social impacts.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a new coding scheme called Stepwise Weighted Spike (SWS) coding scheme for spiking neural networks to enhance the efficiency and reduce the number of operations and thus energy consumption. The SWS coding scheme tackles challenges associated with temporal and rate coding, such as heightened latency and energy usage. It achieves this by compressing spikes and assigning them varying weights at each computational step. Additionally, the paper introduces the Ternary Self-Amplifying (TSA) neuron model, which incorporates a silent phase to mitigate residual errors arising from the weighting procedure.\n\nSTRENGTHS:\nThe SWS coding scheme enhances information capacity and reduces the number of spikes, leading to lower energy consumption and higher accuracy as compared to other coding schemes. The effectiveness of this approach is demonstrated using different datasets.\n\nWEAKNESSES:\n1. Which model of a spiking neuron is being employed in equation 3 (line 120)? What is the reset mechanism here after the neuron fires? Are the weights allowed to have negative values? The description of the model is unclear. \n\n2. The notion of residual error intuitively makes sense but it is confusing. Please define the residual error mathematically (line 139) for better understanding. \n\n3. Why ANN-(sws)SNN conversion is opted instead of directly training the SWS based SNN?\n\n4. There are some recent works [1,2,3] with TTFS encoding which claims better results in regard to energy-efficiency and low-latency. First, these works need to be cited in the related work section. In my opinion, a detailed comparative analysis with other models and encoding schemes (for instance with [1,2,3]) needs to be carried out. \n\n[1] Göltz, J., Kriener, L., Baumbach, A. et al. Fast and energy-efficient neuromorphic deep learning with first-spike times. Nat Mach Intell 3, 823–835 (2021).\n\n[2] I. M. Comsa, K. Potempa, L. Versari, T. Fischbacher, A. Gesmundo and J. Alakuijala, \"Temporal Coding in Spiking Neural Networks with Alpha Synaptic Function,\" ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain, 2020, pp. 8529-8533, doi: 10.1109/ICASSP40776.2020.9053856.\n\n[3] Stanojević, Ana et al. “An Exact Mapping From ReLU Networks to Spiking Neural Networks.” Neural networks : the official journal of the International Neural Network Society 168 (2022): 74-88.\n\nQUESTIONS:\n1. Please address all the weaknesses above. \n\n2. This is an empirical study. Can you provide theoretical proof that the number of time steps or operations required to interpolate a given set of data points (i.e., fitting the training data) is significantly less compared to other methods, such as the Time to First Spike, when using Equation 10 as a metric?\n\nLIMITATIONS:\nThere is no potential negative societal impact and and one limitation related to the inclusion of silent period is noted in the main text.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-08"}

comments: SUMMARY:
The authors introduce a novel encoding method called Stepwise Weighted Spike (SWS) and a corresponding new neuron model named Ternary Self-Amplifying (TSA) for classification tasks utilizing the ANN2SNN training method. The proposed SWS encoding method assigns weights to the importance of spikes at each time step. The TSA neuron, which employs the SWS encoding method, features a lower threshold and includes a silent period.

STRENGTHS:
1. Comprehensive method analysis: the authors conduct a thorough analysis of the Stepwise Weighted Spike (SWS) process, proposing a lower threshold and a silent period method to address residual error issues.

2. Superior Performance: the proposed method demonstrates superior performance in the field of ANN2SNN classification tasks.

WEAKNESSES:
1. Effectiveness of SWS: Various encoding methods, such as rate encoding and Time-to-First-Spike (TTFS) encoding, can be applied to different neurons and models. However, as illustrated in Figure 5, the SWS encoding method alone is ineffective without incorporating a lower threshold and a silent period. It only functions effectively when a neuron employs SWS encoding along with these additional components. Therefore, the paper should emphasize the neuron model rather than the encoding method, as it is not a universally applicable approach.
2. Lack of Experiments: The ablation study shows that the introduction of a silent period is the primary contributor to the improved performance. This raises doubts about the effectiveness of the SWS encoding method itself. Can the authors provide performance metrics for rate encoding combined with a lower threshold and silent period (if applicable) to ensure a fair comparison?

QUESTIONS:
See weaknesses.

LIMITATIONS:
See weaknesses.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-12

comments: SUMMARY:
The authors introduce a new spike coding scheme, which allows them to directly convert quantized ANN to their coding scheme. They demonstrate the effectiveness of their conversion on several pre-trained ANN with minimal loss in performance at the cost of an increase in latency.

STRENGTHS:
- strong experimental results
- coding scheme appears to be novel

WEAKNESSES:
- limited connection to spiking neurons, a more straightforward motivation would be a temporal encoding of quantized ANN

QUESTIONS:
- could you compare your approach more explicitly to ref. 30?

LIMITATIONS:
- method only applicable to conversion from pre-trained ANN
- no demonstration of training of a model using this coding scheme.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-14