SpikePoint: An Efficient Point-based Spiking Neural Network for Event Cameras Action Recognition
Authors: Hongwei Ren
Yue Zhou
Xiaopeng LIN
Yulong Huang
Haotian FU
Jie Song
Bojun Cheng
Venue: ICLR 2024 spotlight (2024)
Keywords: Spiking Neural Betwork
Point Cloud
Event Camera
Action Recognition
Primary area: representation learning for computer vision, audio, language, and other modalities

Abstract: Event cameras are bio-inspired sensors that respond to local changes in light intensity and feature low latency, high energy efficiency, and high dynamic range. Meanwhile, Spiking Neural Networks (SNNs) have gained significant attention due to their remarkable efficiency and fault tolerance. By synergistically harnessing the energy efficiency inherent in event cameras and the spike-based processing capabilities of SNNs, their integration could enable ultra-low-power application scenarios, such as action recognition tasks. However, existing approaches often entail converting asynchronous events into conventional frames, leading to additional data mapping efforts and a loss of sparsity, contradicting the design concept of SNNs and event cameras. To address this challenge, we propose SpikePoint, a novel end-to-end point-based SNN architecture. SpikePoint excels at processing sparse event cloud data, effectively extracting both global and local features through a singular-stage structure. Leveraging the surrogate training method, SpikePoint achieves high accuracy with few parameters and maintains low power consumption, specifically employing the identity mapping feature extractor on diverse datasets. SpikePoint achieves state-of-the-art (SOTA) performance on four event-based action recognition datasets using only 16 timesteps, surpassing other SNN methods. Moreover, it also achieves SOTA performance across all methods on three datasets, utilizing approximately 0.3 % of the parameters and 0.5 % of power consumption employed by artificial neural networks (ANNs). These results emphasize the significance of Point Cloud and pave the way for many ultra-low-power event-based data processing applications.

Published as a conference paper at ICLR 2024
SPIKEPOINT: AN EFFICIENT POINT-BASED SPIKING NEURAL NETWORK FOR EVENT CAMERAS ACTION RECOGNITION
Hongwei Ren, Yue Zhou, Xiaopeng Lin, Yulong Huang, Haotian Fu, Jie Song, Bojun Cheng The Hong Kong University of Science and Technology (Guangzhou)
Jaren066.yyhhu888...o ybuang496..hf31111 coomect..kust-gz.cduuuukk bocchn)) hehusttgz.eduu.cn
ABSTRACT
Event cameras are bio-inspired sensors that respond to local changes in light in- tensity and feature low latency, high energy efficiency, and high dynamic range. Meanwhile, Spiking Neural Networks (SNNs) have gained significant attention due to their remarkable efficiency and fault tolerance. By synergistically harness- ing the energy efficiency inherent event cameras and the spike- processing capabilities of SNNs, their integration could enable ultra-low-pon -oo--eer application scenarios, such as action recognition tasks. However, existing approaches often entail converting asynchronous events into conventional frames, leading to addi- tional data mapping efforts and a loss of sparsity, contradicting the design concept of SNNs and event cameras. To address this challenge, we propose SpikePoint, a novel end-to-end point-based SNN architecture. SpikePoint excels at process- ing sparse event cloud data, effectively extracting both global and local features through a singular-stage structure. Leveraging the surrogate training method, SpikePoint achieves high accuracy with few parameters and maintains low power consumption, specifically employing the identity mapping feature extractor on di- verse datasets. SpikePoint achieves state-of-the-an (SOTA) performance on five event-based action recognition datasets using only 16 timesteps, surpassing other SNN methods. Moreover, it also achieves SOTA performance across all meth- ods on three datasets, utilizing approximately 0.3% of the parameters and 0.5% of power consumption employed by artificial neural networks (ANNs). These results emphasize the significance of Point Cloud and pave the way for many ultra-low- power event-based data processing applications.
INTRODUCTION
Event camera is a recent development in computer vision, and it is revolutionizing the way visual information is captured and processed (Gallego al. 2020). They are particularly well-suited for detecting fast-moving objects, as they can eliminate redundant information and significantly reduce memory usage and data processing requirements. This is achieved through innovative pixel design, resulting in a sparse data output that is more efficient than traditional cameras (Posch et al.) 2010); Son al., 2017). However, most of the event data processing algorithms rely on complex and deep ANNs, which are not aligned with the event camera's low power consumption benefits. Instead, combining event-based vision tasks with SNNs has shown great potential thanks to their highly compatible properties, especially in tasks such as action recognition(Liu et al. 202
Spiking Neural Networks have emerged as a promising alternative that can address the limitations of traditional neural networks for their remarkable biological plausibility, event-driven process- ing paradigm, and exceptional energy efficiency (Gerstner & Kistler 2002). The network's asyn- chronous operations depend on biological neurons, communicating information via precisely timed discrete spikes (Neftei et al. 2019). The event-driven processing paradigm enables sparse but po- tent computing capabilities, where neuron activates only when receives generates spike (Hu
"Corresponding author

---

Published as a conference paper at ICLR 2024
et al. 2021). This property gives the network remarkably high energy efficiency and makes it an idaal candidate for processing event-based data. Nevertheless, existing approaches for combining event cameras with SNN require the conversion of asynchronous events into conventional frames for downstream processing, resulting in additional data mapping work and a loss of sparsity (Kang et al.) (Berlin & John 2020). Moreover, this process leads the loss of detailed temporal informa- tion, which is critical for accurate action recognition (Innocenti et al. 2021). Therefore, developing SNN-compatible novel techniques that operate directly on event data remains challenging.
Point Cloud is powerful representation of 3D geometry that encodes spatial information in the form of sparse set of points and eliminates the need for the computational image or voxel con- version, making it an efficient and ideal choice for representing event data (Qi et al. (2017a). The sparse and asynchronous event data could be realized as a compact and informative 3D space-time representation of the scene, bearing resemblance to the concept of Point Cloud (Sekikawa et al.) 2019]. Still, Point Cloud networks need frequent high-data dimension transformations and complex feature extraction operators in ANNs. Due to their binarization and dynamic characteristics, these may not function optimally in SNNs.
In this paper, we introduce SpikePoint, the first point-based Spiking Neural Network for effec- tively and efficiently processing event data in vision-based tasks. Our contributions are as follows: First, we combine event- based vision tasks with SNN by treating the input Point Clouds rather than stacked event frames to preserve the fine-grained temporal feature and retain the sparsity of raw events. Second, unlike ANN counterparts with multi-stage hierarchical structure, sign singular-stage structure harnessing SNN to effectively extract local and global features. This lightweight design achieves effective performance through the paoo-propagttion training method. Lastly, we introduce a pioneering encoding approach to address relative position data containing negative values within the point cloud. This scheme maintains symmetry between positive and negative values, optimizing information representation. We evaluate SpikePoint on diverse event- based action recognition datasets of varying scales, achieving SOTA results on Daily DVS (Liuu et al. 2021a), DVS ACTION (Miao et al., 2019), HMDB51-DVS datasets, surpassing even tradi- tional ANNs. Additionally, we attain the SNN's SOTA on the DVS128 Gesture (Amir et al. 2017) and UCFI01-DVS dataset (Bi et al. 2020). Notably, our evaluation encompasses an assessment of network power consumption. Compared to both SNNs and ANNs with competitive accuracy, our framework consistently exhibits exceptional energy efficiency, both in dynamic and static power consumption, reaffirming the unequivocal superiority of our network.
de-
RELATED WORK
2.1 EVENT-BASED ACTION RECOGNITION
Action recognition is a critical task with diverse applications in anomaly detection, entertainment, and security. Two primary methods for event-based action recognition are ANN and SNN (Ren ctt al., 2023 The ANN approaches have yielded several notable contributions, including IBM's pioncering end-to-end gesture recognition system (Amir et al. 2017) and Cannici's asynchronous event-based full convolutional networks (Cannici et al. 2019 while Chadha et al. developed a promising multimodal transfer learning framework for heterogeneous environments (Chadha al.) 2019. Additionally, Bin Yin et al. proposed a graph-based spatiotemporal feature learning frame- work and introduced several new datasets (Bi et al. 2020), including HMDB51-DYY
(Ren al.) 2023) proposed an effective lightweight framework to deal with event-based action recognition us- ing a tensor compression approach, and (Shen et al. 2023) proposed an efficient data augmentation strategy for event stream data. On the other hand, SNN methods have also demonstrated great po- tential, with Liu et al. presenting a successful model for object classification using address event representation (Liu et al. 2020) and (George et al.) 2020) using multiple convolutional layers and a reservoir to extract spatial and temporal features, respectively. (Liu et al.) (2211a) have further advanced the field by extracting motion information from asynchronous discrete events captured by event cameras, and (Yao et al. 2023) developed the Refine-and and-Mask SNN (RM-SNN), character- ized by its self-adaptive mechanism modulate spiking responses based on data input. While these SNNs have improved in terms of efficiency, however, their accuracy still falls short when compared to A ANN-based approaches.
2

---

Published as a conference paper at ICLR 2024
Slidnng
Grouping Sumpling
Festure Extreettor
Global Fasture Estracter
ReolT
Cintd
Suen
Heallt
Ref
Rodaj
Boaf
pasat mundect per growp group
The overall architecture of SpikePoint. The raw event cloud is segmented by the sliding window. Then, the global Point Cloud is transformed into M groups by grouping and sampling. The coordinate is converted into spikes by rate coding, and the results of action recognition are obtained by the local feature extractor, global feature extractor, and classifier in turn.
Figure I:
2.2 POINT CLOUD NETWORKS ANN
Point-based methods have revolutionized the direct processing of Point Cloud data input, with PointNet (Qi et al. 2017a) standing out as remarkable example. PointNet++ (Qi et al. 2017b) took it step further by introducing
abstraction module. While it used simple ML. in the fea- ture extractor, numerous more advanced feature extractors have recently been developed to elevate the quality Point Cloud processing (Wu al. 2019 Zhao
2021 Ma et al. 202 Dosovit- skiy et al. 2020 Zhang et al. 2023 Qian et al. 2022 Wu et al. 2023). ). To apply these methods to the event stream, Wang al. (Wang et al. 2019) first tackled the temporal information processing chal- lenge while preserving representation in both the and axes, achieving gesture recognition using PointNet++. PAT (Yang et al.) (019) further improved this model by incorporating self self-attention and Gumbel subset sampling, achieving even better performance in the recognition task. Nonetheless, the current performance of the point-based models still cannot compete with frame-based methods in terms of accuracy. Here, propose SpikePoint as solution that fully leverages the charac- teristics of event clouds while maintaining high accuracy, low parameter numbers, and low power consumption.
3 SPIKEPOINT
3.1
EVENT CLOUD
Event streams are time-series data that record spatial intensity changes in chronological order. Each event be represented by 6m (Im, Vm fmm Pm).. where represents the event number, V.m and Vm denote the spatial coordinates of the event, t. indicates the timestamp the event, and Pm denotes the polarity of the event. It common practice to divide sample Aftaw into some sliding windows A Reti by the next formula to facilitate the effective processing of action recognition data.
AReip clip, (ex-1) V E (1, wwii)) 14 t. L.
where is the length of the sliding window, 1: and represent the start and the end event num- ber of the in sliding window, and nwwin is the number of the sliding window. To apply the Point Cloud method, four-dimensional events in a clip have to be normalized and converted into a three- dimensional spacetime event ARpoint- A straight way to do this is to convert Lm into Zm and ignore
Pm:
A Rgoiint {em (Fmm Mm: 2m) m k.
.
With Zm fm 4i-le
m E (k, and Fm. also normalized between [0, 1]. After pre- processing, the event cloud is regarded the pseudo-Point Cloud, which comprises explicit spatial
3

---

Published as a conference paper at ICLR 2024
information (z, y) and implicit temporal information t. Through pseudo-Poin Cloud, SpikePoint is capable learning spatio-tenne natio-temporal features which is crucial for action recognition.
3.2 SAMPLING AND GROUPING
To unify the number of inputs fed into the network, utilize random sampling ARpoiot construct the trainset and testset. Then, we group these points PN by the Farthest Point Sampling (FPS) method and the K Nearest Neighbor (KNN) algorithm. FPS is responsible for finding the Cen- troid of each group, while KNN is utilized to identify N' group members. This process can be abstracted as follows: Centroid FPS(PN) G= KNNNPN, Centroid, N'
(3)
The grouping of PN into 9 results in transformation of the data dimension from [N, 3] to [N', M, 3j. N is the number input points, M the number of groups, and N' the number of Point Cloud in each group. Given that the coordinates [zt y, x] among the points within each group exhibit similarity and lack distinctiveness, this situation is particularly severe for SNN rate encoding. The standardization method employed to determine the relative position of the point information [Ax, Ay, Az] with respect to their Centroid is as follows:
Az. Ay, A2] G CCenroidd (),, SD(G) E (gi Ø)2
9i E
(4)
SD(g)
Where Arr Ay, Az] adheres to the standard Gaussian distribution NO,, 1). SD corresponds to the standard deviation of G and g = [Zg. Yu. to..... Fn. Yn tn]. Ultimately, we concatenate the relative position Ar, Ay, A2] and Centroid [Te: Ve. ze] the final grouping result.
After grouping, we rate encode the Point Cloud coordinates to meet the SNN network's binary input. It is worth mentioning that the reflected distance information [Ar, Ay. Az] yields positive and negative values, as shown in Fig. 2 While ANN can successfully handle such information due their utilization of floating point operations, SNN employs rate coding and binarization, and can't process negative values. Thus, is necessary to develop a method to effectively handle such information SNN achieve accurate results.
A straightforward approach is to normal- ize [Ax, Ay, A2] [0.1], but this can lead to asymmetric information after passing through points equidistant from the Cen- troid, resulting in limited accuracy. The de- tailed comparison of accuracy is summarized in Table 8 and discussed later. Alternatively, we take the absolute value of the numerator of Eq. too get the |A Alyl, A|2]] that can per- form the spike transformation. However, af- ter such processing, the direction of the rela- tive distance information the coordinates is lost and the distribution of the response data is changed from the standard normal distribution the folded normal distribution. The probability density function is:
(b)
Gaxt)
(bx)
68
(Xite. Zynta
Visualization of our grouping method. (a) The different spatial positions [Ter Nos =e]. [Emin Umin, žmin] and [x; M, 2). (b) The transfor-
Figure 2:
mation the distribution after taking absolute.
(x-p)
(x-1)) 20
f(x; u4 12
(x 2 20)
(5)
2
V
Where y and d are the mean and standard deviation of the Gaussian distribution, respectively. The expectation j after the absolute value given in Eq. erf(z)
/ for e dll is the error function.
2
2 de
1 <(x) " erf( T
+ 1 20(
/2
Eq. 6 We maintain codability and shift the expectation of those coordinates to larger value of which is calculated in Appendix A.1 However, data distribution has also changed
By Eq.
4

---

Published as a conference paper at ICLR 2024
and needs to be compensated. To do so, replace the input [Ax, Ay. Ae, Ze, Bex 2e] mentioned above with [A|z] Alyl. A|2), Zmin- Ymin: Zminl: [mmin:: Dmin 2min] represents the smallest 2, y, values in specific group. Hence, the increase of the first three input components is compensated by the decrease of the last three input components and the input remains balanced. is worth noting that the Centroid in each sphere is important and [Tmin Uminy Eménn] is not good indicator of the Centroid, so we introduce separate branch to extract the global information contained Centroid and do the fusion of these two features in the middle part of the network.
After implementing the aforementioned modifications to the sampling and grouping module, we conducted an analysis that revealed a significant decrease in both the mean relative error (MRE) of rate coding and the coefficient of variation (CV) of the data. This reduction in MRE and CV provides a fundamental explanation for the efficacy of our proposed methodology as evidenced by the results. For the step-by-step step derivation of the formulas and the validation of the dataset, please consult Appendix A.2 and
3.3 SINGULAR STAGE STRUCTURE
The SpikePoint model is unique in its utilization of singular-stage structure as shown in Fig. in contrast to the hierarchical structure employed by all other ANN-based methods for Point Cloud networks. While this hierarchical paradigm has become the standard design approach for ANNs, it is not readily applicable to SNNs because spike-based features tend to become sparse and indistin- guishable as the depth of the stage increases and the training method based on backpropagation has serious gradient problem. In light of this, we develop novel, streamlined network architecture that effectively incorporates the properties of SNNs, resulting in a simple yet highly efficient model adept at abstracting local and global geometry features, The specific dimensional changes and SpikePoint's 's algorithms can be referred to in Appendix A.ppendi(A..aaaa and Algorithm A.5.4
3.3. BASIC UNIT
Next, the basic unit of the feature extractor will be introduced in the form of a formula. The first is a discrete representation of LIF neurons. We abstract the spike into mathematical equation as follows:
S,(1) = Exec, 7(1 s), 7(x) = ((U(L, - Voa)
where S represents the input or output spike, C' represents the set of moments when the spike is emitted, j is the jth input of the current neuron, y represents the spike function, represents the Heaviside step function, Vi denotes the threshold of neuron's membrane potential and = if xl, - Vea 2 0 else 7 = O. We proceed to define the process of synaptic summation in the SNN using the following equation:
A1 [[m] = THnn I/1 1]+ Σ S,
The aforementioned equation delineates the mechanism of neuronal synaptic summation, where the current neuron's 's input is denoted as I, Tyn conforms to the synapse time constant, At represents the simulation timestep, represents the discrete timestep, while refers to the antecedent neuron number.
U + === U[n] [m] [[[]]
where the variable U denotes the membrane potential of the neuron, n represents discrete timesteps, while Tmem the membrane time constant. Moreover, signifies the membrane potential resetting subsequent to the current neuron transmitting the spike.
SNN is prone to gradient explosion and vanishing during training via backpropagation through time (BPTT), owing to the unique properties of their neurons as described above. Drawing on the uti- lization of conventional regularization techniques, such as dropout and batch normalization, we endeavor to incorporate residual module to address the issues of overfitting and detrimental train- ing outcomes. In this work, we achieve identity mapping by modifying the residual module after
neurons in Eq. 10 refer (Hu al. 2021 Fang et al., 202 Feng et al. (022). And the coeffi-
cient (I!+m- + 5!) in Eq. 29 that corresponds to the residual term is canceled out during error
5

---

Published as a conference paper at ICLR 2024
Table Specific information on the five datasets.
DataSet
Classes
Sensor
Avg.length
Truin
Test
Shiding Window
Overlup
DVST28 Gesture
(Amir et al. 2012
10/11
DAVIST28
6.52
26796
6959
0.5
0.25
Daiiy DVS
Liu et al
2021a
12
DAVIS128
3
2924
731
1.58
0.5
DVS Action
Miso et al 2019
10
DAVIS346
5
932
233
0.5 s
0,25
HMDB51-DVS
al.
2020
DAVIS240
8
24463
6116
0.5%
0
UCF101-DVS
Bi at
20Z0
101
DAVIS240
6.68
119214
29803
1s
0.5s
propagation. This coefficient consistently remains below 1, attributed to surrogate gradients. The
accumulative multiplication will cause the gradient to disappear during backpropagation. A detailed derivation can be found in Appendix A.4] which describes how this connection solves the problem.
s = LI 5'-1) + LIF(T) S!
(10)
We have defined two basic units, B and ResF, with and without a bottleneck, respectively, as depicted in Fig. [ and have incorporated them into the overall architecture. Due the speci- ficity of Point Cloud networks, feature dimensioning usually uses one-dimensional convolution. To maintain the advantage of the residual module, connection. As result, the module's 's input and output dimensions remain the same.
not anything to the features of the residual
3.3.2 LOCAL FEATURE EXTRATOR
The local feature extractor crucial abstracting features within the Point Cloud group, similar to convolutional neural networks with fewer receptive fields. As each point is processed, it is im- perative to adhere to the principle of minimizing the depth and width the extractor to ensure the efficiency of the SpikePoint. To this end, we have employed the ResF B unit with bottleneck to streamline the network design as the following equations, result in more advanced extracted features.
X1 a| A yl, A|2], miin: Ymins 2min] X2 [Zes Bec Ze]
(11)
Fi Res F (Cony ID(x))
(12)
F12 Res F F ID (D()))
(13)
Focal = MaxPool Fi)) + Fi2
(14)
We evaluate both the concat and add operations for the two-channel feature fusion in the ablation experiments section.
3.3.3 GLOBAL FEATURE EXTRATOR
The local feature extractor aggregates point-to-point features within each group into intra-group feature vectors, while the global feature extractor further abstracts the relationships between groups into inter-group feature tensors. The input's 's dimension for the final classifier is intimately linked with the width of the global feature extractor. Thus, it is crucial to enable the feature extractor to expand its width much possible while utilizing limited depth, and simultaneously ensuring the extracted features are of high-level abstraction. The feature extractor can be formulated as follows:
L(x) ResF I I(x)))
(15)
Fm = (2( (Ficat)
(16)
F Fglohual MaxPool(Conv1D)
(17)
The final extracted features, denoted as Fgiobut. will be transmitted to the classifier A.7.3 for action recognition. To accommodate small and large datasets, utilized distinct ascending dimen- sionality scales while ensuring consistency in our architecture where specific feature dimensions are illustrated through Appendix Fig.
EXPERIMENT
4.1 DATASET
We evaluate SpikePoint on five event-based action recognition datasets of different scales, and more details about the dataset presented in Appendix A.5 These datasets have practical applications and are valuable for research neuromorphic computing.
4.2 PREPROCESSING
The time set for sliding windows is 0.5 s. and 1.5 shown in Table I The coincidence of adjacent windows is set as 0.25 s in DVS128 Gesture and DVS Action datasets, and 0.5 s in other datasets. The testset is 20% randomly selected from the total samples.

---

Published as a conference paper at ICLR 2024
4.3 NETWORK STRUCTURE
This paper relatively small version network for the Daily DVS and DVS Action, and larger version network for the other three datasets. It should be emphasized that the architectures are identical, with only shift in dimensionality in the feature extraction as presented in Appendix A.7
4.4 RESULTS
DVS128 Gesture: The DVS128 Gesture dataset is extensively evaluated by numerous algo- rithms, serving as benchmark for assessing their efficiency and effectiveness. Compared to the other SNN methods, SpikePoint achieves SOTA results with an
Table 2: SpikePoint's performance on DV$128 Gesture.
Name
Method
Param
Acc
TBR+I3D
Innocenti ct
2021
ANN
12.25 M
99.6%
PointNet++
Qi
20 7
ANN
1.48 M
95.3%
EGCNN
Deny ml
202
ANN
0.82 M
95.7%
VMV-GCN
Xie et al.
2072
ANN
0.86 M
97.5%
Res
Fang
202
SNN
97.9%
accuracy 98. 74% and 98. 1% on class 10 and 11, respectively. Comparing the ANN methods, SpikePoint model demon- strates superior accuracy com- pared to lightweight ANN such as EV-VGCNN and VMV-GCN, despite the latter employing a larger number of parameters, as shown in Table 2 This indicates that Point Clouds' rich and in- formative nature may confer an advantage over voxels conveying complex and meaningful in- formation. The current SOTA employed by ANNs utilizes nearly 21 times more parameters than SpikePoint, resulting
Deep SNN (6 layers) (Amir
2017
SNN
91.8%
Deep SNN(8 layens) (Shresthu & Orchard
2018
SNN
93.6%
Cony RNN SNNIS layers) Xing et
2020
SNN
92.0%
Conv-Reserviir SNN
George a.
2020
SNN
65.0%
HMAX-basee SNN
Liu et
2020
SNN
70.1%
Motion based SNN
Liu et al.
2021
SNN
92.7%
SpikePuint
SNN
0.58 M
98.74%
0.86% increase performance.
Daily DVS: SpikePoint achieves SOTA on this dataset with 97.92% accuracy on the testset, and outperforms all ANN and SNN networks. Notably, it uses only 0.3% of its parameters compared to the high-performing ANN network, as shown in Table 3 Upon visualization of the dataset, we find that it has relatively low noise, and the number of events is evenly distributed in the time domain.
Table 3: SpikePoint's performance Daily DVS.
Name
Method
Acc
Carreii
96.2%
TANet 20211
24.8
96.5%
VMV-GCN (ie 2022
0.84
TimeStormer
Bertasius
ANN
121.27
90.6%
HMAX-based
SNN Xiao
68.3%
HMAX based SNN
2020
SNN
76.9%
Motion-based SNN
Liu ct
2021a
SNN
90.3%
SpikePoint
SNN
0.16
97.92%
The action description well captured by the 1024 points sampled using random sampling. With this in mind, we conduct the ablation study on this dataset, including the processing of negative values (absolute or [O,1]), the utilization of dual channel inputs, and the techniques employed for feature fusion, which are discussed in detail in the Ablation study.
SpikePoint also obtains SOTA re- sults on this dataset, as shown in Table 4 In the same way, we visualize the aedat files in the dataset but find that most of the data have heavy background noise, causing us to collect a lot of noise during random sampling. So we prepro- cess the dataset and finally achieve 90.6% accu- racy. This reflects problem random sampling will not pick up useful points when there is too much background noise. More reasonable sampling will improve the model's robustness.
DVS Action:
Table Model's performance on DVS ACTION.
Name
Method
Parum
Acc
ST-EENet Wann el al. 2020
ANN
1.6 M
88.7%
PointNet
Qi atal
7m
ANN
3.46 M
75.1%
Deep SNNI6 layers)
Gu et
2019
SNN
71.2%
HMAX-based SNN
Xiao et al 2019
SNN
55.0%
Motion based SNN
Liu et
2021a
SNN
78.1%
SpikePPint
SNN
0.16 M
90.6%
HMDB51-DVS: Compared with the first three datasets which have relatively few categories, HMDB51-DVS has 51 different categories. Our experiments demonstrate that SpikePoint excels not only on small-scale datasets but also demonstrates impressive adaptability to larger ones. As shown in Table SpikePoint outperforms all ANN methods, despite using very few parameters.
7

---

Published as a conference paper at ICLR 2024
Table Evaluating IBM gesture's inference dynamic and static energy consumption.
Model
Inpul
Timestep
Accurucy( %)
OPs(G)
Dynamicim.)
Para/Mi
Static(mJ)
SpikePoint
Foint
16
98.7
0.9
0..2
0.58
0.756
2hhng et al. 2021
Frame
40
96.9
4.79
4.36
11.7
15.305
Spikingformer
2023
Frame
98.3
3.72
4.26
3,401
Spikformer Zhou et 2024
Frame
16
97.9
6.33
10.75
2.6
3,401
Deep SNN(16)
Frame
91.8
2.74
2.49
2.223
Deep SNN(8) Shresiina
[SheeJSS de Orchani 2018
Frame
93.6
2.13
1.94
1.7
PLIF (Fang
20216
Frame
97.6
2,98
2.71
17.4
22.759
TBR+13D (eataDDInnnoooo Unnocengi X S N2]
Frame
ANN
99.6
38.82
178.6
12.25
160.23
Event Pramesc1312
Frame
ANN
96.5
30.11
138.5
12.37
16.18
RG CNN
(Miao al
A719
voxel
ANN
96.1
0.79
3.63
19.46
25.45
ACE.BET
ANN
98.8
2.27
10.44
11.2
14,65
VMV GCN
Xic alal
voxel
ANN
97.5
0.33
1.52
1.098
PoinNet++
ANN
95.3
0.872
4.01
1.48
1,936
lepu
kgw
100%
80%
Accuraty
64%
40%
100
2011
300
490
Epoch
Figure 3: Res F ablation experiment (a) and the result (b) on DVS ACTION dataset.
Table 5: Model's performance on HMDBSI-DVS.
This indicates that SpikePoint has excellent generalization capabilities. It should be added that the SpikePoint used for DVS Gesture and HMDB51-DVS is both the larger model and the number of parameter differences are only due to the different categories classified by the final classifier. As the final output layer is a voting layer, the difference in parameter quantities is 0.21 M.
Name
Methed
Param
Acc
C3D
Tran et
2015
ANN
78.41 M
41.7%
13D
Carreira & Ziseemun
2017
ANN
12.37 M
46.6%
ReaNor-34
He et 2016
ANN
63.7 M
43.8%
ResNext-50
Hara ctal.
2018
ANN
26.3 M
39.4%
P3D-63
Qm et 2017
ANN
25.74 M
40.4%
RG-CNN (B 2020
ANN
3.86 M
51.5%
SpikePoint
SNN
0.79 M
55.6%
UCF101-DVS: We continue to test large- scale datasets with 101 categories, which are challenging to model. As illustrated in Table 6, SpikePoint achieves state-of-the-arr results for Spiking Neural Networks with remarkably few parameters, approaching the SOTA results achieved by ANN. These experiments compre- hensively demonstrate that SpikePoint is an ef- ficient and effective model.
Table 6: Model's performance on UCF101-DVS.
Name
Method
Param
Acc
RG CNN +Incep.3D 3D Bi al. 2020
ANN
6.95M
63.2%
B3D Carreira & Zisserman
2017
ANN
12.4M
63.5%
ResNext 50 Hara et al. 2018
ANN
26.05M
60.2%
ECSNet-SES (Chen
2022
ANN
70.2%
Res Bes-SNN-IT Fang al. 2021
SNN
57.8%
RM-RES-SNN- 18 Yao
2023
SNN
58.5%
SpikePoint
SNN
1.05M
68.46%
4.5 POWER CONSUMPTION
Spikepoint's power consumption exhibits substantial advantages over ANNs and SNNs as shown in For details calculating power consumption, please refer to A.9
Table 7
4.6 ABLATION STUDY
ResF ablation: To verify the efficacy of the proposed feature extractor ResF, conduct ablation experiments on the DVS ACTION dataset by varying only the feature extractor variable, keeping the overall architecture, hyperparameters, and datasets consistent. As depicted in Fig. a)), three groups of experiments are conducted: the first group utilizes the residual structure with the same architecture as ANN, the second group uses the SpikePoint model without residual connection, and the third group employs the SpikePoint. The results of experiments are marked with blue. orange, and gray, respectively, Fig. B(b). The curves demonstrate the superiority of the ResF block, and the results zoom in the accuracy curves the beginning and stabilization periods of the model. SpikePoint converges fastest at the beginning training and has the highest accuracy after stabilization. Compared with the model without residual, can be proved that this module
8

---

Published as a conference paper at ICLR 2024
Network Structure
100%
Locil
Cheiitrr
Accurucy
60%
40%
20%
Locdl
cLUd DrI
Chadfeer
0%
0
100
200
300
400
Epoch
Figure 4: Structural ablation experiment (a) and the result (b) DVS ACTION dataset.
Table 9: Ablation study SNN's timesteps.
Time steps
2
8
12
16
24
32
DVS Acc. (%)
92.75
95.17
96.53
97.22
97.92
96.7
96.01
DVS Action Acc.(%)
60.93
71.88
80.09
85.65
90.6
88.39
81.03
can improve the model's s fitting ability. SpikePoint is better convergence speed and convergence accuracy than copying the ANN scheme.
Structural ablation:
Table 8: Ablation study on grouping in Daily DVS dataset.
In the structure ablation experi- ment, we conduct four compari- son experiments. The first group involves the standard SpikePoint. The second group is the network with only local feature extrac- tor. The third group focuses 8
No.
Absolute
[mmin...
Branch
Fusion
Performance
single
97.22%
2
0,
x
single
96.53%
3
0.1
X
single
97.36%
X
single
97.63%
single
97.78%
double Add
97.92%
V
X
double Concal
97.50%
X
double Add
97.50%
solely on global feature extrac- tion of the Point Cloud set with- out any grouping or sampling op- erations, as shown in Fig. 4a). The fourth group is the SNN version of PointNet, which with an architecture that essentially identical to the third group except for differing dimensions of [32, 64, 128, 256] and [64, 128, 256, 512, 1024), respectively. The results provide evidence of the effectiveness of combining local and global features. As depicted in Fig. 4b), the gray and orange lines represent architectures with only global and local feature extractors, respectively. Not surpris- ingly, these models perform poorly on the DVS Action dataset, with an accuracy of only 40%. To compare with PointNet, we adjust the dimensionality of each stage to be consistent with it, and the yellow color indicates the resulting training curve. We observe that the model shows relatively better results when the dimensionality reaches 1024.
9
X
X
double Add
97.22%
[0,1]
double
Add
96.25%
The timestep serves as a crucial hyperparameter, profoundly influencing the ac- curacy and power consumption in SNN. The choice of the key parameter is result of the controlled experiment with varying timesteps. We do seven sets of comparison experiments on Daily DVS and DVS Action respectively, and the results are shown the accuracy of the testset in Table
Timestep ablation:
To overcome the challenge of rate encoding input with negative values, we introduce five relevant variables, each corresponding the number of columns in Table & We then conducted several experiments to verify their validity. The results demonstrate the effectiveness of taking absolute values for input, which are superior to normalization, with 2 and 5 in Table And [Eimnn; Vmninn Emin]] can have certain corrective effect with 2 and 3, 6 and 8. However, Centroid affects the results more than [Imin Ymin, Zmin] in the single channel with and 5. Another branch is used compensate for this phenomenon with 4 and 6. The dual path structure is better than the single path with and 6, and experiments have shown that the Add operation better with 6 and 7.
Grouping ablation:
CONCLUSION
In this paper, we introduce full spike event-based network that effectively matches the event cam- eras' data characteristics, achieving low power consumption and high accuracy. SpikePoint singular-stan stage structure is capable of extracting both global and local features, and it has demon- strated excellent results five event-based action recognition datasets using propaaropagttion but not converting ANNs to SNNs. Going forward, our aim is to extend the applicability of SpikePoint to other fields of event-based research, such SLAM and multimodality.

---

Published as a conference paper at ICLR 2024
6 ACKNOWLEDGEMENT
This work was supported in part by The Hong Kong University of Science and Technology (Guangzhou) Joint Funding Program under Grant 2023A03J0154 and Grant 2023A03J0013, well as from the Young Scientists Fund of the National Natural Science Foundation of China (Grant 62305278).
REFERENCES
Arnon Amir, Brian Taba, David Berg, Timothy Melano, Jeffrey McKinstry, Carmelo Di Nolfo, Tapan Nayak, Alexander Andreopoulos, Guillaume Garreau, Marcela Mendoza, et al. A low power, fully event-based gesture recognition system. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7243-7252, 2017.
S Jeba Berlin and Mala John. R-stdp based spiking neural network for human action recognition.
Applied Artificial Intelligence, 34(9):656-673, 2020.
Gedas Bertasius, Heng Wang, and Lorenzo Torresani. Is space-time attention all you need for video understanding? In ICML, volume 2, pp. 4, 2021.
Yin Bi, Aaron Chadha, Alhabib Abbas, Eirina Bourtsoulatze, and Yiannis Andreopoulos. Graph- based spatio-temporal feature learning for neuromorphic vision sensing. IEEE Transactions on Image Processing, 29:9084-9098, 2020.
Marco Cannici, Marco Ciccone, Andrea Romanoni, and Matteo Matteucci. Asynchronous convolu- tional networks for object detection in neuromorphic cameras. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 0-0, 2019.
Joao Carreira and Andrew Zisserman. Quo vadis, action recognition? a new model and the kinetics dataset. In proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp- 6299-6308, 2017.
Aaron Chadha, Yin Bi, Alhabib Abbas, and Yiannis Andreopoulos. Neuromorphic vision sensing for cnn-based action recognition. In ICASSP 2019-20199 JEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.
IEEE, 2019.
Zhiwen Chen, Jinjian Wu, Junhui Hou, Leida Li, Weisheng Dong, and Guangming Shi. Ecsnet: Spatio-temporal feature learning for event camera. IEEE Transactions on Circuits and Systems for Video Technology, 33(2):701-712, 2022.
Yongjian Deng, Hao Chen, Huiying Chen, and Youfu Li. Ey-vgenn: A voxel graph cnn for event- based object classification. arXiv preprint vrXiv:2106.0 02066000166 16, 2021.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, al. An im- age is worth 16x16 words: Transformers for image recognition at scale. International Confer- ence on Learning Reprexentations, 2020.
Wei Fang, Yanqi Chen, Jianhao Ding, Ding Chen, Zhaofei Yu, Huihui Zhou, and Yonghong Tian. other contributors. spikingjelly, 2020.
Wei Fang, Zhaofei Yu, Yanqi Chen, Tiejun Huang, Timothée Masquelier, and Yonghong Tian. Deep residual learning in spiking neural networks. Advances in Neural Information Processing Systems, 34:21056-21069, 202
Wei Fang, Zhaofei Yu, Yanqi Chen, Timothée Masquelier, Tiejun Huang, and Yonghong Tian. In- corporating learnable membrane time constant to enhance learning of spiking neural networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2661-2671, 2021b.
Lang Feng, Qianhui Liu, Huajin Tang, De Ma, and Gang Pan. Multi-level firing with spiking ds-resnet: Enabling better and deeper directly-trained spiking neural networks. arXiv preprint arXiv:2210.06:2 2022.
10

---

Published as a conference paper at ICLR 2024
Yang Feng, Hengyi Lv, Hailong Liu, Yisa Zhang, Yuyao Xiao, and Chengshan Han. Event density based denoising method for dynamic vision sensor. Applied Sciences, 10(6):2024, 2020.
Guillermo Gallego, Tobi Delbrück, Garrick Orchard, Chiara Bartolozzi, Brian Taba, Andrea Censi, Stefan Leutenegger, Andrew J Davison, Jörg Conradt, Kostas Daniilidis, et al. Event-based vision: A survey. IEEE transactions on pattern analysis and machine intelligence, 44(1):154-180, 2020.
Arun M George, Dighanchal Banerjee, Sounak Dey, Arijit Mukherjee, and Balamurali. A reservoir-based convolutional spiking neural network for gesture recognition from dvs input. In 2020 International Joint Conference on Neural Networks (IJCNN), pp. 1-9. IEEE, 2020.
Wulfram Gerstner and Werner M Kistler. Spiking neuron models: Single neurons, populations, plasticity. Cambridge university press, 2002.
Pengjie Gu, Rong Xiao, Gang Pan, and Huajin Tang. Stca: Spatio-temporal credit assignment with delayed feedback in deep spiking neural networks. In IJCAI, pp. 1372-1372, 2019.
Kensho Hara, Hirokatsu Kataoka, and Yutaka Satoh. Can spatiotemporal 3d cnns retrace the history of 2d cnns and imagenet? In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 6546-6555, 2018
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Mark Horowitz. 1.1 computing's energy problem (and what
do about it). In 2014 IEEE international solid-state circuits conference digest of technical papers (ISSCC), pp. 10-14. IEEE, 2014.
Yangfan Hu, Huajin Tang, and Gang Pan. Spiking deep residual networks. IEEE Transactions Neural Networks and Learning Systems, 2021.
Simone Undri Innocenti, Federico Becattini, Federico Pernici, and Alberto Del Bimbo. Temporal binary representation for event-based action recognition. In 2020 25th International Conference on Pattern Recognition (ICPR), pp. 10426-10432. IEEE, 2021.
Ziyang Kang, Lei Wang, Shasha Guo, Rui Gong, Shiming Li, Deng, and Weixia Xu. Asie:
An asynchronous snn inference engine for aer events processing. ACM Journal on Emerging Technologies in Computing Systems (JETC), 16(4):1-22, 2020.
Chang Liu, Xiaojuan Qi, Edmund Lam, and Ngai Wong. Fast classification and action recognition with event-based imaging. IEEE Access, 10:55638-5564) 2022.
Qianhui Liu, Haibo Ruan, Dong Xing, Huajin Tang, and Gang Pan. Effective aer object classification using segmented probability probabiity-mmximizzatee maximization learning in spiking neural networks. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp. 1308-1315, 2020.
Qianhui Liu, Dong Xing, Huajin Tang, De Ma, and Gang Pan. bvent-based action recognition using motion information and spiking neural networks. In IJCAI, pp. 1743-1749, 2021a.
Zhaoyang Liu, Limin Wang, Wayne Wu, Chen Qian, and Tong Lu. Tam: Temporal adaptive module for video recognition. In Proceedings of the IEEE/VVF international conference on computer vision, pp. 13708-13718. 2021b.
Xu Ma, Can Qin, Haoxuan You, Haoxi Ran, and Yun Fu. Rethinking network design and local geometry in point cloud: A simple residual mip framework. In International Conference on Learning Representations, 2021.
Shu Miao, Guang Chen, Xiangyu Ning, Yang 7i, Kejia Ren, Zhenshan Bing, and Alois Knoll. Neu- romorphic vision datasets for pedestrian detection, action recognition, and fall detection. Frontiers in neurorobotics, 13:38, 2019.
Emre O Neftci, Hesham Mostafa, and Friedemann Zenke. Surrogate gradient learning in spiking neural networks: Bringing the power of gradient- based optimization to spiking neural networks. IEEE Signal Processing Magazine, 36(6):51 1-663, 2019.
11

---

Published as a conference paper at ICLR 2024
Adam Paszke, Sam Gross, Francisco Massa, Adam Lever, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high- performance deep learning library. Advances in neural information processing systems, 32, 2019.
Christoph Posch, Daniel Matolin, and Rainer Wohlgenannt. A qvga 143 db dynamic range frame- free pwm image sensor with lossless pixel-level video compression and time-domain eds. IEEE Journal of Solid-State Circuits, 46(1 ):259-275, 2010.
Charles R Qi. Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 652-660, 2017a.
Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas Guibas. Pointnet++: Deep hierarchical fea- ture learning point sets in metric space. Advances in neural information processing systems, 30, 2017b.
Guocheng Qian, Yuchen Li, Houwen Peng. Jinjie Mai, Hasan Hammoud, Mohamed Elhoseiny, and Bernard Ghanem. Pointnext: Revisiting pointnet++ with improved training and scaling strategies. Advances in Neural Information Processing Systems, 35:23192-23322 922244,4 2022.
Zhaofan Qiu, Ting Yao, and Tao Mci. Learning spatio-temporal representation with pseudo-3d residual networks. In proceedings of the IEEE International Conference Computer Vision, pp- 5533-5541, 2017.
Hongwei Ren, Yue Zhou, Haotian Fu, Yulong Huang, Renjing Xu, and Bojun Cheng. Tipoint: A tensorized point cloud network for lightweight action recognition with event cameras. arXiv preprint arXiv:2308.0999 2023.
Shikha Saun and Hemant Kumar. Design and performance analysis of 6t sram cell on different cmos technologies with stability characterization. In IOP conference series: materials science and engineering, volume 561, pp. 012093. IOP Publishing. 2019.
Yusuke Sekikawa, Kosuke Hara, and Hideo Saito. Eventnet: Asynchronous recursive event process- ing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3887-3896, 2019.
Guobin Shen, Dongcheng Zhao, and Yi Zeng. Eventmix: An efficient data augmentation strategy for event-based learning. Information Sciences, 644:1191775 2023.
Sumit B Shrestha and Garrick Orchard. Slayer: Spike layer error reassignment in time, Advances in neural information processing systems, 31, 2018.
Bongki Son, Yunjae Suh, Sungho Kim, Heejae Jung, Jun-Seok Kim, Changwoo Shin, Keunju Park, Kyoobin Lee, Jinman Park, Jooyeon Woo, al. 4.1 640x 480 dynamic vision sensor with 9um pixel and 300meps address-event event representation. In 2017 IEEE International Solid-State Circuits Conference (ISSCC), pp. 66-67. IEEE, 2017.
Du Tran, Lubomir Bourdev, Rob Fergus, Lorenzo Torresani, and Manohar Paluri. Learning spa- tiotemporal features with 3d convolutional networks. In Proceedings of the IEEE international conference on computer vision, pp. 4489-4497, 2015.
Qinyi Wang, Yexin Zhang, Junsong Yuan, and Yilong Lu. Space-time event clouds for gesture recog- nition: From rgb cameras to event cameras. In 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1826-1835. IEEE, 2019.
Qinyi Wang, Yexin Zhang. Junsong Yuan, and Yilong Lu. St-evnet: Hierarchical spatial and tempo- ral feature learning space-time sinee-time event clouds. In Proc. Ady. Neural Inf. Process. (NeurlPS)) 2020.
Wenxuan Wu, Zhongang Qi. and Li Fuxin. Pointcomv: Deep convolutional networks on 3d point clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- fion, pp. 9621-9630, 2019.
12

---

Published as a conference paper at ICLR 2024
Yue Wu, Jiaming Liu, Maoguo Gong, Zhixiao Liu, Qiguang Miao, and Wenping Ma. Mpet: Multi- scale point cloud transformer with a residual network. IEEE Transactions on Multimedia, 2023.
Rong Xiao, Huajin Tang, Yuhao Ma, Rui Yan, and Garrick Orchard. An event-driven categorization model for aer image sensors using multispike encoding and learning. IEEE transactions on neural networks and learning systems, 31(9):349933657 2019.
Bochen Xie, Yongjian Deng, Zhanpeng Shao, Hai Liu, and Youfu Li. Vmv-gen: Volumetric multi- view based graph cun for event stream classification. IEEE Roboties and Automation Letters, (2):1976-1883, 2022.
Yannan Xing, Gaetano Di Caterina, and John Soraghan. A spiking convolutional recurrent neural network (sernn) with applications to event-based hand gesture recognition. Frontiers in neuroscience, 14:590164, 2020.
Jiancheng Yang, Qiang Zhang, Bingbing Ni, Linguo Li, Jinxian Liu, Mengdie Zhou, and Qi Tian. Modeling point clouds with self-attention and gumbel subset sampling. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 3323-3332. 2019.
Man Yao, Hengyu Zhang, Guangshe Zhao, Xiyu Zhang, Dingheng Wang, Gang Cao, and Guoqi Li. Sparser spiking activity can be better: Feature refine-and-mast spiking neural network for event-based visual recognition. Neural Networks, 166:410-423 423, 2023.
Renrui Zhang, Liuhui Wang, Yali Wang, Peng Gao, Hongsheng Li, and Jianbo Shi. Starting from non-parametric networks for 3d cloud analysis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5344-5353, 2023.
Hengshuang Zhao, Li Jiang, Jiaya Jia, Philip HS Torr, and Vladlen Koltun. Point transformer. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pp. 16259-16268, 2021.
Hanle Zheng. Yujie Wu, Lei Deng, Yifan Hu, and Guoqi Li. Going deeper with directly- trained larger spiking neural networks. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pp. 11062-11070, 2021.
Chenlin Zhou, Liutao Yu, Zhaokun Zhou, Han Zhang, Zhengyu Ma, Huihui Zhou, and Yonghong Tian. Spikingformer: Spike-driven residual learning for transformer- spiking neural net- work. arXiv preprint arXiv:2304. 11954, 2023.
Zhaokun Zhou, Yuesheng Zhu, Chao He, Yaowei Wang, Shuicheng Yan, Yonghong Tian, and Li Yuan. Spikformer: When spiking neural network meets transformer. arXiv preprint 27X1v:2209.15425. 2022.
13

---

Published as a conference paper at ICLR 2024
A APPENDIX
A.1 THE SHIFT OF μ
By taking the absolute values, the normalized data becomes all-positive distribution, and the corresponding PDF is adapted as follows:
(x-
f (1; 2
(E
25t
+
28
(x 2 0)
(18)
The expectation of the new distribution can be obtained by integrating:
x f(:; u1, ,1, ydr
(19)
The data after standardization as obeying the standard Gaussian distribution~NNO.1)), bringing in J = and = 1:
22 (e= 1e +++ +e )dr
2 V
/ π Ju
xe dx
2 V _- e
= d(
2
(20)
2T
The actual shift the Daily DVS dataset is 0.75, which aligns closely with the estimated value. However, there remains slight discrepancy as normalization does not alter the underlying distribu- tion of the original data. If the original data deviates from a normal distribution, the standardized data will not fully conform to normality either. The original data points hold geometric signifi- cance, acquired through FPS and KNN methods, yet they do not necessarily adhere to a Gaussian distribution. Given input Ar. Ay, Az, Fe: Yo, =c], want the network to locate the points within the group by [ + Ar. Ye + Ay, 2e + Az). But we rescale the data, so the expression becomes [xte A Yo Ay SD, %e + A SD.. When the first three [Az, Ay, Az] increase, we want the whole to remain the same, so we should decrease the last three [xe, Me. ze]. Consequently, we choose
to utilize [Emin: Umin: 2min] rather than precisely correcting the Centroid (centroid - V SD). On the Daily DVS dataset, these two methods achieve testset accuracies 97.92% and 97.50%, respectively.
A.2 MRE IN RATE CODING
The error in the transformation process due to the SNN coding using as little as 16 timesteps to describe the 128, 240, and 346 resolution data is not negligible. As matter of fact, using such small timestep, within each group of points, the points often become indistinguishable. To better differentiate the points within group, scale of the coordinate system from d, to Åd, Before and after the rescaling, the Mean Relative Error (MRE) can be expressed by Eq. 21 and Eq. 22] respectively:
apply rescaling step by Eq. 4 to change the origin and
x(dx,) |d:ll
|d E |g Centroid
(21)
dd
1=1
1SD DT j= X ((6)
|dlll
d,
8pjda1
A|d
(22)
n
[d
SD
where 8 denotes MRE, T represents the timesteps in SNN, SD the standard deviation of G, n indicates the number of points, and |d] refers to the Manhattan distance of each point in a group to the Centroid, where the dimension of |d] is [N', M,3). We reshape into one-dimensional to calculate MRE, and (1, M -3). The stateless Poisson encoder represented by the function X.
For instance, in Daily DVS dataset, SD ~ 0.052, |d,| 0.039, and A A|d, |d, is the average of [d,] which from all points. As we expected, A |dg| is very close to the theoretical
d
0.75. Where
SD
value (v 2) we calculate in Eq. 6 Next, through experiment we get 8 d. RJ 0.26 and 6/2,1 2 1.07.
Our method results in 76% reduction in encoding error of the coordinates.
14

---

Published as a conference paper at ICLR 2024
A.3 COEFFICIENT OF VARIATION
In this paper, a stateless Poisson encoder is used. The output |d is issued with the same probability the value d be encoded, where P((d,) |dil; |d,] [. and |d,1 } Σ x(Id;)). After performing the rescaling operation, the data, that are not at the same scale, the coefficient of variation (CV) can eliminate the influence of scale and magnitude and visually reflect the degree of dispersion of the data. The definition of CV is the ratio of the standard deviation to the mean, n represents the number of trials, follows:
SD CU
SD((d,))
(d,)2 x 1dgl
X
T |d,1 (1 (4)) x x
(23)
n X
m((d)) = ald, = - Σ
Idils
(24)
jml
CU (d.l)2 x |d,1 |d,1 (1 (d:1) (ald;1)²
a aeldil + |d,12 20 |d
(25)
(a (o))
Where a is the scale factor, and p((dg)) is an unbiased estimator of the expected value. If the number of trials n is large, then a follows normal distribution (1, (. - T is the timestep of SpikePoint, which is 16, n is the number of times the same value is encoded, the input dimension of the network is [N' .,, 6] = [1024, 24, 6]. the resolution of the value is related to the DVS, for example, Daily DVS 128, so the average number of trials to each resolution 1 1152. We simulated the rate coding with the above data to derive a ~
0.0116), which is narrow normal distribution. Approximately, if we bring a 1 into Eq. 25 to get concluded that the larger [dg] is, the smaller the coefficient of variation is, and the network is more easily able learn the knowledge of the dataset. This improvement is reflected in the accuracy of the trainset of Daily DVS, which increases from 95% before conversion to 99.5% by replacing [d,] to A A.
/1/ld,1 1. From the formula, it can be
A.4 RESIDUAL IN BACKPROGATION
The residual module in ANN is expressed by the following equation, A is the state vector of the neurons, and B is the bias vector of the neurons:
A' = W B¹, o0='(AA" + x1-2)
(26)
Awl 1j = aL aw! = -A
x WTd + (A(ttm- + +Y)) Σ 3 k
(27)
k
When the output of the above formula A' is O, the value of Y is the value of the input Y is nonlinear function. This operation is known as identity mapping. Eq. 27 represents the rule of backpropagation of gradients with residuals, L represents the loss function, represents the parameters of the (th layer, o is the derivative of the activation function, yl-1 represents the input of the 7th layer of the network, 8 represents the error of output neuron k. m represents the output of the current layer after the residuals to the m layer. If we apply this module directly in SNN, this will cause the problem of gradient explosion or vanishing (Fang al. 202 We can do identity mapping by changing the residual module to the following form in SNN. And the coefficient (1(+m + 5t)) in Eq. 29 of error propagation of the corresponding residual term is canceled.
s LIF(1 + S1-1) - LIF(I) + S
(28)
AW
OW. 19
aL = o' (1)s
-1
8 WT +o ( +s Σ
gl
%
W T.
(29)
15

---

Published as a conference paper at ICLR 2024
o(x) 1 x
arctan (Ta) + 2' d'(x) (1 (TTZ)²
(30)
Due to the non-differentiable nature of the function, the surrogate gradient method is often used when training SNN. If 5 is equal to 1, the expectation of 7!+m-1 + S is around 1 because 7(+m-1 is the output of batch normalization layer, the expectation of the surrogate gradient a + (t) which is calculated by Eq. is always less than one. This will cause the error propagated through the residual to become smaller and smaller until it disappears. The modified residual connection can effectively alleviate this phenomenon.
A.5 EXPERIMENT DETAILS
We evaluate the performance of SpikePoint on our server platform, during both training and test- ing. Our experiment focuses on measuring the accuracy of the model and examining the size of model parameters across different datasets. The specific specifications of our platform include an AMD Ryzen 9 7950x CPU, an NVIDIA Geforce RTX 4090 GPU, and 32GB of memory. For our implementation, we utilize the PyTorch (Paszke et al. frameworks.
2019) and spikingjelly (Fang al.) 2020)
Table 10: More details in five action recognition datasets.
DataSet
Resolution
Samples
Record
Denoise
SpikePoint
DVS128 Gesture
128x128
1342
direct
x
large
Daliy DVS
128x128
1440
direct
small
DVS Action
346x260
450
direct
small
HMDB51-DVS
320x240
6776
convert
large
UCFI01-DVS
320x240
13320
convert
large
A.5.1 DATASET
The details of the datasets are shown in Table 9. The DVS 128 Gesture dataset is collected by IBM (Amir et al.] (017). comprising 10/11 gesture classes. It is captured with DAVIS 128 camera under different lighting conditions with a resolution of 128x 128. The Daily DVS dataset (Liu et al. 2021a) contains 1440 recordings of 15 subjects performing 12 daily actions under different lighting and camera positions. It is also recorded with DAVIS 128 camera. The DVS Action dataset (Mio al. 2019) features 10 action classes with 15 subjects performing each action three times. It is recorded with a DAVIS 346 camera with a resolution of 346x240. The HMDB51-DVS (Bi) al. 2020) dataset includes 51 human action categories recorded with conventional frame-based camera recordings and subsequently converted into events using DAVIS 240 conversion mode. The resolution 320x240. Each of those datasets has its own application scenario, and all of them are widely used for research and benchmarking in neuromorphic computing. We visualize some samples from the four datasets understand event- based action recognition better, shown in Fig. 5
A.5.2 NOISE
There are two main types of noise in the event data, illumination noise (hot pixels) and background noise. Illumination noise pertains the noise introduced by changes in ambient lighting conditions during data capture, and background noise is the generated event from the unwanted movement of the background. These two types of noise are represented in all four datasets. For the illumination noise, the DVS Action dataset exhibits noticeable illumination noise, depicted in Fig. 6]a). That noise is likely introduced during the recording, as the illumination noise could be present in certain scenarios. The illumination noise profoundly impairs the effectiveness of the network during random sampling. Consequently, we use the denoising method (Feng et al., 2020) to reduce the noise in the sample and the probability of sampling points from noise. As for the remaining three datasets, their illumination noise is relatively low, so we proceeded directly. As for background noise, HMDB51- DVS indicates strong presence of background noise. As illustrated in Fig. b)), the label is hugging
16

---

Published as a conference paper at ICLR 2024
(a)
DVS ACTION
Jamping
Resolution:
346x260
Sit down
(b)
Daily
Run
Resolution: 128
(c)
IBM GESTURE
Air guitar
Resolution:
128 128
clockowise
(d)
HMDBB 51
Resolution: 320 x 240
Figure 5: Visualization of four datasets. (d)HMDB51-DVS.
(a)DVS (b)iily DVS.(CIIM
Gesture.
(6) Illemination noise (Hot pixel)
(b)
Backgroand/Distracti Distranttnttttttt noise
[c) Ideal sample for sampling
Jomping
Hug
Figure 6: Visualization of different noise. (a) The schematic of illumination noise (b) The schematic of background noise (c) The representation of ideal sample
17

---

Published as a conference paper at ICLR 2024
Action
Timestamps
Actioa
Action
Start
Ead
Figure 7: Visualization of sit sample from DVS Action. We selected the second half of the event stream to build the dataset.
people, and the red area is drawn for the background of the building. Since most of the Point Clouds sent into the network originated from background buildings, the network can not recognize the action. To our best knowledge, this problem remains challenging in the community. The background noise significantly limits the accuracy of the action recognition the HMDB51-DVS dataset.
A.5.3 LABEL
In the DVS Action dataset, there is no detailed timestamp setting, only including the action start and end labels. After we visualize the event stream the files, as depicted in Fig. 7 we find that almost all actions occur after half of the total dataset. So set the sampling start point from half of the total timestamps to build the dataset. Compared with the manual annotation method (Wang et al. 2020), our method is straightforward but still achieves better results on the DVS Action dataset.
A.5.4 SLIDING WINDOW
Algorithm The algorithm of SpikePoint
The appropriate length of the sliding window affects the performance the network. We initially used sliding : Clip the event stream; window of 0.5 s in Daily DVS, and the
Data: ARfaw Event cloud data from event cameras Result: Categories of action recognition
: while not the end of event stream do
accuracy of the model was only 80%
Obtain the start ta and end fú;
with this setting. After comparative experiments,
Get the sliding window clip ARaspi
finally optimized the end value with 1.5 s, representing the aver- 6 if nuwn then age length of the 12 actions identified 7 in the Daily DVS dataset. Determin- ing the appropriate length of the slid- ing window for describing an action is valuable task to explore in the future, for instance, automated adjustment win- dow.
Randomly sampling from clip;
Convert clip to PN ; Normalized datac [0. 1]
end
Grouping;
Find the Centroid through FPS for group in Centroid do
g (Centrrl, N', .NN): Get [|a] Alyl, A|2l, Tmins Donins Rmin);
iS
A.6 OVERALL ARCHITECTURE
end
The pipeline which is shown in Fig. begins with the receipt of the original 19
Rate coding for coordinates,the timestep is T;
Point Clouds generated by event cam- for i in T do eras. A sliding window clip of the origi- nal data in the time axis is performed by employing the method discussed in Sec- tion 3.1 Subsequently, a random sam- 24 pling of the Point Clouds in the sliding end
Extract features:
Sequential([for block in local extractor) & [for block in global extractor].[MP))
Classifier:
Sequential(fe,, bug, lif,, feg, bug lify, votinglayer)
window is conducted to create a point set of dimensions [.V. 3]. The sam- pling and grouping method highlighted in Section 3.2 is then applied to process
the set and derive group of dimensions [N', M, 6). The Point Cloud dimension is subsequently
18

---

Published as a conference paper at ICLR 2024
Local Feature Extractor
Feature
Clussifer
piOL
pitts oq
piss ca
Convld
Czas Ie
Redf
Coartd
Vie
RoitB
SpikcPuint specifii
Informationn Duily DVS,
POLETO P SPAL
N: poier sumber per greup
mumber
(b)
Local Feuture Extraciee
Global Feacure Instracter
Classtfer
Coocil
Comrld
SpilccPoint speeifie dimensiendl Information Gester.HDDBBS
N: paist number per group
dara dineetioon
Figure Two models of different sizes corresponding to the four datasets. In addition, each Conel is followed by a batch normalization layer, and the bottleneck of ResFB is half of the input dimension.
up-dimensioned by employing multi-local feature extractor, which results in tensor of dimensions [N M, D). The local features then reshaped and pooled to obtain tensor of dimensions [M, D'J. J. representing the global features. The global features are then extracted utilizing global feature extractor. The global and local features are abstracted into a single feature vector of dimensions [1, D" by employing the max pooling operation for classification by the classifier.
A.7 NETWORK
A.7.1 STRUCTURE
In this work, we keep the same network architecture for all the datasets. Nevertheless, to accommo- date datasets of different sizes, the dimensionality of the network in the feature extraction process is adapted. A relatively small network was used for the Daily DVS and DVS Action, and a relatively large network was used for the other two datasets. The details of the small and large networks are highlighted in blue and red Fig. g We use the list to represent the Point Cloud's 's dimension change of two different size models, [32, 64. 128, 256] and [64, 128, 256, 512]. The first represents the change in the dimensionality of the input Point Cloud data by the local feature extractor. The remain- ing three represent the global feature extractor's up-dimensioning of the local feature dimension. In addition, the width the classifier is [256, 256, numfiass * 10) and [512, 512, num * 10) in that or- der. Due the smaller dimensions output by the global feature extractor compared to other models, the dropout layer is omitted between the extractor and classifier.
A.7.2 LIF OR IF
We find that LIF neurons have better performance results the DVS128 Gesture dataset than IF neurons. They are 94%74 for LIF neurons vs. 97.78% for IF neurons. We speculate that the overfitting of the model is somewhat alleviated due the leaky of the neurons.
A.7.3 CLASSIFIE
The classifier aligns with the majority of network solutions by flattening the features and forwarding them to spike-based based MLP with numerous hidden layers for classification. However, instead of the number of neurons in the last layer being equal to the number of categories, a voting mechanism used vote with ten times the spike output. Thus, the number of neurons in the classifier's last
19

---

Published as a conference paper at ICLR 2024
layer is ten times the number of categories for classification. We squared error (MSE) as the loss function of SpikePoint. The equation is as follows:
num-
Loss
Σ
Σ
(
(Y
(31)
T
The model predicts the result as Y and the actual label as y. The encoding timestep of the SNN model is T, num is the Action recognition category for the dataset, and the loss function is shown above.
A.8 HYPERPARAMETERS
We utilize rate encoding to encode coordinates, and the timestep is set to 16. The neuron model we employ is ParamericLIF (Fang al., (02Tb)) whose initial is 2.0, and decay input. In backpropagation, we use the ATan gradient for surrogate training (Neftci et al. 2019).
Our training model uses the following hyperparameters, Batch Size: 6 or 12, Number of Points: 1024, Optimizer: Adam, Initial Learning Rate: Le 3, Scheduler: cosine, Max Epochs: 300.
A.9 POWER CONSUMPTION DETAILS
We assume that multiply-and-accumllaa (MAC) and accumulate (AC) operations are implemented on the 45 nm technology node with VDD = 0.9 V (Horowitz) 2014). where EMAC 4.6 pJ and EAC 0.9 pJ. We calculate the number of synaptic operations (SOP) of the spike before calculating the theoretical dynamic energy consumption by SOP = firerate x T x FLOPs, where T is the timesteps, and FLOPs is float point operations per sample. OPs in Table 7 refer to SOPs SNNs and FLOPs in ANNs. The dynamic energy is calculated by Dynamic = OPs x EMAC or Ac and static energy is calculated by Stalic Pura. spp Lsamples where spp the static power per parameter in SRAM and Taample is the sample time length, assuming always-on operation. The static power consumption of 1iit SRAM is approximately 12.991 pW in the same technology node (Saun & Kumar 2019).
20


Reviews:
[{"review_id": "uFgM6SdYH1", "reviewer": "Reviewer_XXP9", "summary": "This paper proposed a novel and efficient network approach for event based action recognition. The network leveraged spike neural network as backbone. The preprocessing of the events includes grouping, sampling and rate coding to feed in spike format. The grouping takes special consideration to avoid asymmetric information pass-through. The proposed approach also has shown improving the mean relative error and coefficient of variation. \n\nThe SNN learns from both the point cloud centroids and the processed representations. The feature learning part contains both local and global feature extractors as well as residual connection to avoid weight explosion/vanishing.\n\nThe approach has been tested on various datasets including small and large ones. The paper has also compared with SOTA methods for similar tasks. \n\nThe proposed approach has significantly low power consumption, especially compared to other non SNN based networks. The results are strong and the advantages are salient.", "questions": "None.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "The paper has proposed several novel processing steps accompanied by theoretical derivations. The paper first looked at how to convert the events into SNN acceptable format. One of the issues is that directly normalizing delta positions will result in asymmetric information passthrough. The paper calibrated this offset by using the delta of the absolute values. In the SNN part, the paper incorporated residual learning modules to prevent weight explosion/vanishing. \n\nThe performance of the proposal has been demonstrated on several datasets and has strong improvement over existing methods.", "weaknesses": "I don't find notable weaknesses. I only find the proposed methods could also be extended to other relevant tasks, which this paper has deferred to future work. Otherwise, I think the paper results are pretty solid.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AzddBWX9gf", "reviewer": "Reviewer_NnWo", "summary": "This paper proposes an SNN framework for event stream processing, termed SpikePoint. It first processes the event stream into point groups and encodes using the rate coding method. Then, the local and global feature extractors are proposed to learn the deep features based on spiking activation neurons.\n\nthe writing of this work needs further polishment; a lot of typos can be found all through the paper;\nthe idea of pure snn for event point stream processing is not new; as the key components are all off-the-shelf modules;\nthe experiments on large-scale event-based recognition datasets are missing; which is hard to judge whether the proposed method works.", "questions": "1. further polish this paper; \n2. re-organize the contributions of this work, as the current version does not shown significant difference with existing works;\n3. more experiments on large-scale event datasets are needed.", "limitations": "", "rating": 3, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "This paper proposes an SNN framework for event stream processing, termed SpikePoint. It first processes the event stream into point groups and encodes using the rate coding method. Then, the local and global feature extractors are proposed to learn the deep features based on spiking activation neurons.", "weaknesses": "the writing of this work needs further polishment; a lot of typos can be found all through the paper;\nthe idea of pure snn for event point stream processing is not new; as the key components are all off-the-shelf modules;\nthe experiments on large-scale event-based recognition datasets are missing; which is hard to judge whether the proposed method works.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2D3mR7miV6", "reviewer": "Reviewer_qCNh", "summary": "In this study, the authors present a spiking neural network tailored for event-based action recognition, utilizing event cloud data. The designed network adeptly captures both global and local features. Notably, the introduced method sets new benchmarks by achieving state-of-the-art results on four distinct event-based action recognition datasets.", "questions": "Please refer to 'Weaknesses'.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The proposed method is novel and interesting. \n\nThe proposed method achieves sota performances on four event-based action recognition datasets.", "weaknesses": "1. It is suggested to explain why employing the ResFB in the local extractor and the ResF in the global extractor.\n\n2. Regarding the experiments conducted on DVS Gesture, please specify whether the setting encompasses 10 classes or 11.\n\n3. For clarity in Table 1, it would be more efficient to consolidate all pertinent information within a single row.\n\n4. Could you clarify the term \"Single-stream\"? Based on Figure 1, the entire network appears to consist of two distinct streams.\n\n5. In the related work section, consider incorporating more contemporary research related to both 'event-based action recognition' and 'point cloud network in ann'.\n\nMinor issues:\n\nThere's an inconsistency in the experimental outcomes for SEW-Resnet as presented in Table 2 and Table 6.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dx3M2M0AJg", "reviewer": "Reviewer_tiMk", "summary": "The authors propose a spiking neural network that applies to event-driven camera output and is applied to action detection (agents moving in the visual scene). The authors show that their method achieves performance comparable to state-of-the-art methods, but with significantly lower latency and energy consumption.", "questions": "In addition, I think the paper could be improved by the following points:\n\n- Numerous methods have been developed in the past to study dynamic scenes, such as particle importance sampling, and in particular the \"condensation\" method by Isard and Blake. What parallels do you see between your method and these methods?\n- In Table 7, you show that performance is optimal for a given number of time steps... What can you deduce from this result in relation to the complexity of the data representation?\n\nMinor:\n- \"C represents the set of moments\" - you mean instants?\n- The point \"A detailed derivation can be found in Appendix A.4, which describes how this connection solves the problem of backpropagation.\" is vaguely introduced, please describe minimally the method in the main text.\n- The syntax of the paper did not allow me to fully follow all arguments. I have not taken this into account in my evaluation, but the authors should use a service, even an automatic one, that allows clarification of certain points. Fix for instance \"bionic neurons\" > \"biological neurons\" or vague statements like \"to harmoniously extract local...\", . Also check the sentence \"We do identity mapping by changing the residual module to the following equation in SNN refer (Hu et al., 2021; Fang et al., 2021a; Feng et al., 2022). And the coefficient σ′ (Il+m−1 + Sl ) in Eq. 29 of error propagation of the corresponding residual term is canceled.\n- The LaTeX formatting of the paper could be improved. In particular, quotations in the text should be enclosed in parentheses, e.g. using `citep`. Text appearing in equations (\"erf\", \"clip\", \"centroid\", \"lif\", ...) should be formatted as text, e.g. using `\\text``.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The method is clearly presented in the paper and is built around the use of point clouds, which are used to represent events. The method uses a relatively classical global architecture that consists in extracting local features, in order to group them intermediately to form a representation that will be efficiently processed by a final classification layer. Overall, the paper is well written and the results are clearly presented.", "weaknesses": "A major argument of the paper is to propose a method that deals directly with events that are constituted by the output of an event camera. The authors' argument is to be able to transform events into point clouds and thus improve network performance: \"SpikePoint, is an end-to-end point-based SNN architecture\". However, the figure shows that after the grouping and sampling stage, the information is transformed by coding the firing rate: \"The coordinate is converted into spikes by rate coding, and the results of action recognition are obtained by the local feature extractor, global feature extractor, and classifier in turn\". This point needs to be clearly justified, and in particular why isn't the temporal information kept precisely at this point in the processing process. Is that information rather represented in the previous stages?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (spotlight)