Spiking Neural Network as Adaptive Event Stream Slicer
Authors: Jiahang Cao
Mingyuan Sun
Ziqing Wang
Hao Cheng
Qiang Zhang
shibo zhou
Renjing Xu
Venue: NeurIPS 2024 poster (2024)
Keywords: Event-based Camera
Spiking Neural Network
Object Tracking
Image Recognition
Primary area: neuroscience_and_cognitive_science
TL;DR: A novel-designed event processing framework capable of splitting events stream in an adaptive manner.

Abstract: Event-based cameras are attracting significant interest as they provide rich edge information, high dynamic range, and high temporal resolution. Many state-of-the-art event-based algorithms rely on splitting the events into fixed groups, resulting in the omission of crucial temporal information, particularly when dealing with diverse motion scenarios (e.g., high/low speed). In this work, we propose SpikeSlicer, a novel-designed event processing framework capable of splitting events stream adaptively. SpikeSlicer utilizes a low-energy spiking neural network (SNN) to trigger event slicing. To guide the SNN to fire spikes at optimal time steps, we propose the Spiking Position-aware Loss (SPA-Loss) to modulate the neuron's state. Additionally, we develop a Feedback-Update training strategy that refines the slicing decisions using feedback from the downstream artificial neural network (ANN). Extensive experiments demonstrate that our method yields significant performance improvements in event-based object tracking and recognition. Notably, SpikeSlicer provides a brand-new SNN-ANN cooperation paradigm, where the SNN acts as an efficient, low-energy data processor to assist the ANN in improving downstream performance, injecting new perspectives and potential avenues of exploration.

Spiking Neural Network as Adaptive Event Stream Slicer
Jiahang Caol* Mingyuan Sun²" Ziqing Wang Hao Cheng'
Qiang Zhang'4 Shibo Zhou Renjing Xu
The Hong Kong University of Science and Technology (Guangzhou)
Northeastern University Northwestern University
Beijing Innovation Center of Humanoid Robotics Co. Ltd. Brain Mind Innovation
caco248, hcheng046, qzhang749) Ccommmecttttcc hkust-ga edu. cn mingyuansun200gmai.. com, ziqingvang2029eu. nowthwestern...mmm..morrrr.. edu bob@brain-mind. com. cn, renjingcuhhkust-g edu.
Abstract
Event-based cameras are attracting significant interest they provide rich edge in- formation, high dynamic range, and high temporal resolution. Many state-of-the-ard event-based algorithms rely on splitting the events into fixed groups, resulting in the omission of crucial temporal information, particularly when dealing with diverse motion scenarios (e.g., high/low speed). In this work, we propose SpikeSlicer, a novel-designed plug-and-play event processing method capable of splitting events stream adaptively. SpikeSlicer utilizes a low-energy spiking neural network (SNN) trigger event slicing. To guide the SNN to fire spikes at optimal time steps, propose the Spiking Position-awee -warre Loss (SPA-Loss) to modulate the neuron's state. Additionally, develop Feedback- Update training strategy that refines the slic- ing decisions using feedback from the downstream artificial neural network (ANN). Extensive experiments demonstrate that method yields significant performance improvements in event-based object tracking and recognition. Notably, SpikeSlicer provides brand-new SNN-ANN cooperation paradigm, where the SNN acts efficient, low-energy data processor to assist the ANN in improving downstream performance, injecting new perspectives and potential avenues of exploration. Our code is available at https::/ / github...... com/ aadyCao1125/$pikeel.ccc /Spikekkiicer....
1125
seeettttin Aibvrt
Event slicing by fixed duration
Spene Tesker wit
Tracking
Event slicing by Fixed count AN
Evemh-canurra
Recagnition
Downstream
Adaptive event sticing SNN
adaptive
adaytive
Event representation
Figure 1: Comparison of event slicing methods. Traditional methods slice event streams based on prefixed time intervals (a) or event counts (b). In contrast, our approach (c) utilizes SNN as a dynamic event processor for adaptive event slicing. The sliced sub-event streams be converted into various event representations with robust information and then applied to multiple downstream tasks.
'Equal contribution.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Introduction
Event-based cameras [1] are bio-inspired sensors that capture event streams in an asynchronous and sparse way. Compared with conventional frame-baser cameras, event-based cameras offer numerous outstanding properties: high temporal resolution (with the order of μs), high dynamic range (higher than 120 dB), low latency, and low power consumption. Over recent years, rapid growth has been witnessed in dealing with event data due to the inherent advantages of event-based cameras, such as object tracking [2, 3]. depth estimation [4, 5], and recognition [6, 7]. Before applying to various downstream tasks, the event stream must be split by groups and then transformed into different event representations, e.g., frame, voxel, or point for deep learning architectures.
conversion consists of two steps: (1) slicing the raw event stream into multiple sub-event stream groups, and (2) converting these sub-event streams into different event representations, Much of the current research focuses on the second step, aiming to refine event representation [8] techniques such as time surface [6] and event spike tensor [9]. Yet, this focus often overlooks the crucial first step of slicing, where issues such as the non-uniformity of the information contained in the fixed-sliced event remain unaddressed.
In detail, the process of event-to-representaaiie
To address the challenges in the slicing process, we delve into the limitations of traditional slicing techniques. Common methods typically cut the event stream into several fixed groups. For example, slicing event stream with fixed event count [10] fixed time intervals [11, 12] depicted in Fig. 1. However, these fixed-group slicing techniques often lead to problems: they may cause insufficient information capture in low-speed motion scenarios or excessive redundancy in high-speed conditions, thereby failing to accurately capture the dynamic changes in event distribution. Additionally, some pyper-parameters. e.g., the length of time interval, are highly-sensitive (examples are provided in Appendix C) and must be carefully pre-deternined. Although some latest slicing methods [13, 14] propose to adaptively sample the events, there still exists the problem of hyper-parameter tuning which can not achieve fully learnable and adaptable slicing process.
the downstream tasks
In order address the above issues, we propose SpikeSlicer, novel-design event processing method that can adaptively slice the event streams. To achieve this, SpikeSlicer utilizes an SNN as an event trigger dynamically determine the optimal moment to split the event stream. Our objectives include: (1) training the SNN to spike at a specific time step for accurate event slicing, and (2) developing training strategy to identify the best slicing time for a continuous event stream during training. In our paper, we achieve (1) through our newly introduced Spiking Position-aware Loss (SPA-LosS) function, which effectively guides the SNN spike at the desired time by manipulating the membrane potential. For (2), implement receives real-time performance feedback from the downstream ANN model for supervision. An overview of our proposed method is depicted in Fig. 2. We evaluate the effectiveness of our proposed SpikeSlicer in two downstream tasks: (i) event- event-based object tracking. which is strongly sensitive to temporal information and motion dynamics, and (ii) weent-based recognition, which is highly related to event density. Extensive experiments validate the effectiveness of the proposed approach.
Seedback-Update training strategy, where the SNN
To sum up. our contributions
follows:
We propose SpikeSlicer, a novel and-aayy event processing method capable of splitting event streams an adaptive manner.
We design the SPA-Loss to guide the SNN to trigger spikes at the expected time steps. We then propose novel Feedback-Update strategy that optimizes the event slicing process based on the ANN feedback.
Extensive experiments demonstrate that SpikeSlicer significantly improves model perfor- mance by up to 1 11.9% in object tracking and 19.2% in recognition with only 0.32% increase energy consumption.
2 Background and Related Work
Event-based Cameras. They are bio-inspired sensors, which capture the relative intensity changes asynchronously. In contrast to standard cameras that output 2D images, event cameras output sparse event streams. When brightness change exceeds threshold C, an event e is generated containing
position u (xr, y), time 4t, and polarity Ps: (T 4x) = I(u, (e) - T(u, 4 Al+) = C.C. C. The polarity of an event reflects the direction of the changes (i.e., brightness increase ("ON") or

---

Spike
Syike Nexron
00:00
WVNNT atream
Object Tracker
event group
everd representation
Recogriber
downstrcam ANN
prediction
label
00100
2-5
opika aequeece
Lapá
0
Feedback Loss
desired spite
SNN
Adappiva Event Slking Process (Ag
Pondhach- Stralega (4lp-)
Figure 2: Overview of our method. The input events are first fed into an SNN, and the event is determined to be sliced when a spike occurs. To find the accurate slicing time, the neighborhood search method explores other time steps. and feeds event representations to the downstream ANN model (e.g., object tracker or recognizer). The ANN model then offers feedback, which guides the SNN in firing spikes at the optimal slicing time by supervising the membrane potential through the Spiking Position-awe -warre Loss CSPA.
decrease ("OFF").. In general, the output of an event camera is a sequence of events, which can be described as: & = {ex 1x { & Pk. pe I== With the advantages of high temporal resolution, high dynamic range, and low energy consumption, event cameras are gradually attracting attention in the fields of tracking [3, 15], identification [7], 3D reconstruction [16, 17] and estimation [5].
Spiking Neural Network (SNN). SNNs are potential competitors to artificial neural networks (ANNs) due to their distinguished properties: high biological plausibility, event-driven nature, and low power consumption. In SNNs, information is represented by binary time series data rather than float representation, leading to energy efficiency gains. Also, SNNs possess powerful abilities to extract
features for various tasks, including recognition [18], tracking [2], and image generation [19]. In this paper, we adopt the widely used Leaky Integrate- tantte--n--irr Fire (LIF, [20, 21]) model, which effectively characterizes the dynamic process spike generation and can be defined as:
Vin] = 3V [[m,,
(1)
S(n] e [n] Oa.).
(2)
where is the time step and 3 is the leaky factor that controls the information reserved from the previous time step; V[n] is the membrane potential; S(ni] denotes the output spike which equals I when there is a spike and otherwise; ((x) is the Heaviside function. When the membrane potential exceeds the threshold on. the neuron will trigger spike and resets its membrane potential V Visset < Ssh. Meanwhile, when 3 = y = LIF neuron evolves into integrate-and-Fi
(IF) neuron. We also introduce no-reset membrane potential Un/]. meaning that the membrane potential does not reset, but directly passes the original value to the next time step (i.e., U'n] V[n] after Eq. 2).
Our Approach: SpikeSlicer
In this section, we first introduce the concept of event cells for data preparation (Sec. 3.1). We then introduce the adaptive event slicing process by utilizing an SNN as the event trigger (Sec. 3,2). In Sec. 3.3, we introduce a novel Spiking Position-aware Loss (SPA-Loss) to supervise the SNN slice the event at the precise time. Lastly, we build a feedback-update (Sec. 3.4) strategy that allows the resulting events to be correlated with the feedback from the downstream model, thereby improving overall performance.
3.1 Converting Event Stream to Event Cell
Event streams are asynchronous data that can be represented as a set: & {[x,, ye, fv. p.I... with time span of I (i.e., t E [fo, to + T)). We envision ideal situation where the SNN is utilized to directly process the raw data stream and slice the event. However, in software simulations, the event stream should be converted into event representations to comply with the input requirements of existing deep learning frameworks. In this paper, following the mainstream research, we adopt the voxel-grid representation [22] as the input of SNN. We first introduce the definition of event cell:
Definition (Event cell). Consider a small time interval 8t, event cell is a single-grid event representation in the form of: C+ (z. V,l,) = FvozelGG (x. L, {L E [la, +50))) where F F denotes the voxel grid [22] method to process the raw event groups G (Appendix F) with [t., ty + 6t) into a grid representation.
3

---

A whole event stream can be then represented by a list of N event cells, i.e., [C,(x, y, to), (a y. + št).... C1 (r. y, to + (N - }1) where T/ôt and each cell corresponds to discrete time index E {0, 1, ... ,N 1}. The mapping of discrete time to real event time interval is defined as:
fume = {t|t E [to + nót, to + (n + 1)6t}}
e.g., the time interval of (x. y, to) is (t) In the following sections, we abbreviate the event cell as Clrl].
Discussion:
Distinct from the typical voxel grid, an event cell only contains brief time interval, i.e. öt small. At this point, the entire cell sequence appropriately represents the raw event stream, while simultaneously fulfilling the input requisites for the SNN.
3.2 Adaptive Event Slicing Process
Utilizing SNNs neuromorphic hardware for processing event streams is low-energy and low- latency [23, 24). Hence, we adopt the Spiking Neural Network as the event stream slicer, aiming for dynamically slicing the event stream enhance the downstream performance. Incorporating with the SNN, we now describe the adaptive event slicing process:
Considering an event stream &, we first convert into a list of continounsu event cells. Event cells are then continuously entered into SNN (structure details are provided in Appendix L) through a loop operation. Through forward propagation, the features of the last hidden layers (h--1) are finally mapped to single spiking neuron to activate spikes:
Sont = LLI((SNNro(A
Once the spiking neuron generates spike (i.e., Sout = 1) at current time Tee is considered a slicing action. This allows to obtain the time interval from the end of the previous spike to the current spike. Suppose the previous spike happened at time ny. the real event time interval is within Tgroup = - Jtime (ni {l (lo + (Mp 1)81, to + (ne 1)8()} Then, the raw event data within this time interval form an event group, which can be converted to any event representation:
Dn, F (,,, Tgroup));
where F denotes an event representation method (e.g., frame [22], point [25], graph [26] and surface [6]-based methods). This representation D.,, is then prepared for the downstream tasks.
The slice of the event stream is determined by the state (excited/resting of the dynamic event trigger, SNN promptly decides to split events upon spike generation and extracts the precise time interval of the raw event stream. The details of the adaptive event slicing process is shown Alg.
Event Slicing Rule:
SNN's spiking neuron. Serving
3.3 Spiking Position-aware Loss
In this section, we propose the Spiking Position-awart Loss (SPA-Loss). which contains two parts: (1) membrane potential-driven loss (Mem-Loss) is used directly guide the spiking state of the spiking neuron at a specified timestamp, and (2) linear-assuming loss (LA-Loss). which is designed to resolve the dependence phenomenon between neighboring membrane potentials, achieving more precise spiking time. Moreover, we introduce a (3) dynamic hyperparameter tuning method to avoid the experimental bias caused by the manual setting of hyperparameters.
3.3.1 Membrane Potential-driven Loss
As mentioned the previous section, the slice position of event is determined upon the spike occurrence. The challenge now lies in directing the SNN trigger spike precisely at the optimal position, once the label for this optimal slicing position is provided (in Sec. 3.4).
Consider consecutive event cells inputs starting from the previous spiking time, suppose we expect SNN to slice the event at n", i.e., spike Sout is triggered at n². This corresponds to the membrane potential of the spiking neuron needing to reach the threshold VnA at n=", which inspired us to guide the spike time by directly giving the desired membrane potentials. However, membrane potential returns to the resting state immediately after the occurrence of a spike, which may result inaccurate guidance at later moments (Appendix G). Thus, choose to supervise the no-reset membranc

---

Membrane Potential Gap
Expected
TransT on FE108
Expected
DIMP on FEIU8
Mem.
Time 24
Time 22
2.2
comont weking
2.1
Viu
2.0
Mon
seperiinn
1+2
0-1
n11
Time
25 50 Epoch
50 Epoch
(b)
Figure 3: Empirical observations: (a) Hill effect in adaptive slicing process; (b) Impact of hyperpa- rameter a settings on TransT tracker [27] and (c) DiMP tracker [28].
potential U[n]] (Eq. 16) to exceed the threshold (i.e., U(n") 2 Vax). The membrane potential-driven loss defined as:
Camm ||0 ||Unn"] (1 a)
(6)
where 2 0 is hyperparameter to control the desired membranc potential to exceed the threshold. However, an excessively high a may directly induce a premature spike in the neuron, thereby influencing the membrane potential state at the targeted time step. We provide a proposition to address this problem:
Proposition 1. Suppose the input event cell sequence has length N, desired spiking time is n (n' E {0, N.., - 1}), the membrane potential at time n² satisfying the constraints:
Ví Unn"] m +
(7)
where I [n*) is the imput synaptic current from Eq. Then the spiking neuron fires spike time n' and does not excite spikes at neighboring moments.
The proof is provided in the Appendix H. Based on the proposition, modify the loss function into:
Mem || [nn"]] a + at uuUuper)]]]
(8)
where Ulower Vin and Uupper max + <[[n"], Vo.) denote the lower and upper bounds of the U [n"] provided the proposition, respectively; a E [0, 1] balances the desired membrane
potential U between Ulower and Uppper- Experiments in Sec. 4.1 demonstrate that Mem-Loss is able to supervise the SNN to determine the slicing of the event flow at specified timestamp. More details of Mem-Loss are provided in Appendix I.1.
3.3.2 Linear-assumine Loss
However, only using Mem-Loss is unable guarantee that the spiking neuron trigger spikes any expected timestamp. We have the following observations:
Observation (Hill effect). Suppose there exists situation where S[n] 1 and U [n] 2 Un + 1). If the neuron expected to activate a spike at time n + 1, Unn + 1] will be driven to reach the threshold through the Mem-Loss. Nonetheless, the supervised neuron still exhibits Umm] 2 Uin 1]. causing an early spike at time n.
As illustrated in Fig. 3(a), Uf > Unn + 1] exists, this membrane potential gap is still inherited after the supervision. In this case, when the later membrane potential is guided to exceed the threshold, the earlier membrane potential reaches the threshold sooner and turns the neuron into the resting state. This poses challenges obtaining a second spike at the later moment.
Therefore, we expect the later membrane potential to increase monotonically with the time step to reverse the hill effect. Here we use the simplest linear monotonically increasing assumption to construct the loss function:
[|U[dd]]
Vin The 12. if condition,;
CLA
0.
otherwise,
where denotes the expected spike timestep and nh denotes the current spike timestep, condition, corresponds to the nonlinear monotonically increasing condition that satisfies: U[ne]] 2
5

---

Algorithm 1 Adaptive Event Slicing Process
Algorithm 2 Feedhack-Updat' Training Strategy
Iuput: SNN model, input event E. the number of event cell N. the previous spike index uy. list Drop-
SNN model, pretrained ANN model, ANN training dataset A.. SNN training dataset Ds. total epoch Eerain, epoch to start finctuning es alpha a, leamning rate 1).
Input:
Initialize: " =0 D.
for all 0, 2...... do
for all = 1,2, Ereato epach do
Sant SNN(C((1))
for event batch d, = dy, da, ...dx, D5
Sout then
Food d, into SNN until spikes at time stop ne- Generate
candidates to ANN get feedhack. Calculate loss function Caps = Lmen + LLA- Backpropagate and update SNN parameters. end for
T Tgroup = Unamp Seima (n).
E Erouup (2. 2, Tacoup)
Dng F(Egroup):
2. > (n n///N, ey then
Append Dn. to Drep.
Ro +
Split D A into 2'A by adaptive event slicing process. Finetune ANN parameters D'a-
end if
end lor
end end for
Return: Drey-
U[n"] & ne < n². We expect the membrane potential at og to reach "e Vin for the latter membrane potential at the n" to reach Via in a linearly increasing form. More explanations are provided in Appendix I.2.
Combined with Mem-Loss and LA-Loss, we defined the SPA-Loss, which guides the adaptive event slicing process subsequent experiments: LSPA Leem + LLA
3.3.3 Dynamic Hyperparameter Tuning
Although controlling the SNN to spike at a desired location can be achieved through the combination of Mem-Loss and LA-Loss, the utilization of varying a values (Eq. 8) may result in significant fluctuations in experimental results. We have the following observation.
Observation 2. The larger the a, the earlier the SNN tends to fire spikes; and vice versa.
larger a in Eq. 8 implies higher pre- rr-moometarry membrane potential, which results in an earlier spike. Taking the larger-a scenario in Fig. 3(c) as an example, if the SNN is expected to activate a spike at a later timestep, the larger a prevents the actual spike from being delayed. Thus, we need to decrease a, causing the expected spike time to shift earlier. This concludes that the update direction of the hyperparameter a should be consistent with the update direction of the desired spiking index.
Observation 3. A fixed a leads to significant variations in performance across different tasks.
As illustrated in Fig. 3(b) and (c), the same varies significantly different downstream models, which makes it difficult to set the hyperparameter alpha in advance.
To address the above issues, we design learning- based hyperparameter tuning method for updating a (in Alg. 2). More details are provided in Appendix I.3.
3.4 Feedback-Update Strategy through SNN-ANN Cooperation
Based on the methods proposed in the previous sections: if the desired trigger time n² is given, the SNN is able to accurately accomplish the event slicing under the guidance of the SPA-Loss function. In this section, we focus on how to obtain the desired spike time n through the downstream ANN feedback. We thus propose feedback-update strategy that enables the SNN slice events when the downstream ANN model achieves optimal performance. By receiving real-time feedback from the downstream model and updating n²", this strategy ultimately enhances task performance.
Particularly, when SNN processes the input event and triggers a spike at time n. it returns a spike output sequence $ = [0....0, 1, ...,..., where 1 is at ng-th. We first perform neighborhood search to obtain 2d + 1 candidate event representation with the index in {ne d..... n2 + d} {Dn,-d= .... Dn.+d), ), where Dr.++ F( y, tt ft [to + (n, 1)ōt, to + (n, + 1 + ))) We then choose a downstream model A (e.g., object tracker or recognizer) and feed the candidate event representations into it to obtain feedback yy:
y (C - d)) ... L.((Cnn d))
(10)
where (-M(-) returns the output loss of M and + concatenates these losses into y E 22++1 We choose the model loss as the feedback since it directly reflects the quality of inputs. We can then generate the desired spike index n by: n²" = arg myl(). where arg min extracts the index with

---

3,4
800
gets
ene
5.2
600
Mom
5.0
400
Splie MSE Los Spila
4.8
4.6
10
10
20
30
Training Epoch
Desired Spiking Time Step
(a)
(b)
Figure 4: (a) Experiments on comparing different loss functions on simple event slicing task. Our proposed Mem-Loss and LA-Loss require only a small number of iterations to supervise the SNN to activate spikes at the desired time steps; (b) Experiments on different hyperparameter settings. Our dynamic tuning method can stably converge towards the optimal spiking time (colored in green). In contrast, using a fixed results in unstable training and challenges finding the optimal point.
the best (minimal loss) feedback, which in turn guides the dynamic slicing process using SPA-Loss. After training the SNN, the ANN is then updated by feeding the newly-sliced events, thus forming an SNN-ANN cooperation process.
Feedback-Update Strategy. This strategy employs a two-stage iterative approach. In the first stage, the ANN offers real-time feedback to the adaptive slicing process for training SNN. In the second stage, the trained SNN provides a newly-sliced event to finetune the ANN. The process then iterates back to the first stage. This strategy provides novel SNN-ANN cooperation paradigm which establishes a strong connection between raw data and the downstream model. We summarize the feedback-un update strategy in Alg.
4 Experiments
To evaluate the effectiveness of our proposed method, we set up two-level experiments. In the beginner's is arena, expect the SNN to find the exact slicing time with the simulated event inputs. In the expert's arena, we conduct experiments on event-based object tracking and image recognition. Details of experiment settings and more experimental analyses are presented in the Appendix.
4.1 Beginner's s Arena: Event Slicing in Simple Tasks
We first conduct some entry-level tasks to validate the effectiveness of SPA-Loss. We set up the task: Input N randomized event cells, expect the SNN to slice a specified time step and there exists certain probability of interfering with SNN to slice at other time steps.
We compare the SPA-Loss function with common MSE-Loss and CE-Loss. The experiment setting detailed Appendix K. depicted in Fig 4(b), the SPA-Loss successfully supervises the SNN to activate spikes at the desired time steps. In particular, SPA-Loss requires only few iterations ( (<400) to supervise the SNN fire spikes correctly. steps, and CE-Loss cannot even accomplish the task. In addition, using both Mem-Loss and LA-Loss yields smoother results compared to using Mem-Loss alone. To summarize, the beginner's 's arena preliminarily tests the effectiveness of the SPA-Loss and paves the way for subsequent experiments.
contrast, MSE-LLss can only succeed at certain time
4.2 Expert's Arena: Mastering Adaptive Event Slicing with SNN-ANN Collaboration
After successful challenge in the beginner's 's arena, we move on to the expert arena. Here we use low-energy SNN to adaptively process the event data on complex downstream tasks:
Event-based Object Tracking. Since the tracking task is highly sensitive temporal informa- tion, dynamic event slicing is of great importance. We provide two versions for adaptive slicing: SpikeSlicer-Base (B) and SpikeSlicer-Small (S). The detailed consumption of these two versions are introduced in the ablation study. Tab. 1 shows that the tracking performances under the SpikeSlicer have a significant improvement terms of representative success rate (RSR), representative precision rate (RPR), and overlap precision (OP). For instance, TransT with SpikeSlicer-S" performance on OPo.5n and OP7.75 improved by 5.8% and 3.2% compared to its original result under the HDR scenario. When compared to results on the fixed event (the number of the fixed-sliced event is aligned with the number of dynamic-sliced events to ensure fair comparison), our method achieves favorable gains in the overall RSR, i.e., 63.6 vs. 51.0.
7

---

Table I: Quantitative comparison FE108. There are four challenging scenarios, including high dynamic range (HDR), low light (LL), fast motion with and without motion blur (FWB & FNB) and all testing datasets (ALL). [method]++ SSpieeSiieiiieeeeeee--B. represents the results based our adaptive event slicing method with base (B) or small (S) SNN version. The results of [method] are reproduced on the original fixed-sliced event dataset. To ensure fair comparison, fix event indicates that the model is tested on a dataset of fixed-sliced event frames, where the number of fixed event frames is the same as the number of dynamically sliced event frames by using SpikeSlicer. Best performances are denoted by deep green.
HDR
LL
FWB
FNB
ALL
Methods
RSR
O0
75 RPR
RSR
OP
50 OP.75
RPR
RSR
OP 30
OP 75
RPR
KSR
OP.50
50
75
RPR
RSR
OP
s0
OP.7 76 RPR
SiamFC++ [29]
15.3 15.0
1.3
15.3
28.6
42.7
7.4
63.1
23.8
26.0
39.T
KYS [30]
15.7 14.5
5.2
23.0
12.0
8.0
1.1
18.0
47.0
63.9
14.8
73.3
36,9
44.5
15.2
57.9
26.6
30.6
9.2
41.0
CLNett [31]
30.0
33.5
9.6
48.3
13.7
6.0
0.9
23.6
52.9
71.2
23.3
80.3
40.8
46.3
14.2
67.7
34.4
39.1
11.8
55.5
DiMP [28]
49.1 60.3
16.3
77.1
67.3
87.4
40.4
96.9
52.5
53.9
7.8
98.5
50.0
60.1
21.4
78.2
52.1
62.4
17.9
84.3
DiMP (fxed event)
53.3 68.2
21.4
86.3
43.1
95.0
49.7
45.4
5.88
49.6
59.4
23.7
75.3
53.8
82.4
DiieseppiceS
53.3 67.3
21.9
79.8
69.8
92.3
47.2
96.5
64.1
83.4
27.9
97.2
54.4
67.4
27.7
81.2
57.3
73.0
25.8
86,1
DiMP4SpikeSlleer-S
56.0 72.0
27.0
80.7
70.0
92.2
50.6
95.0
66.7
86.1
38.4
96.9
56.4
70.6
31.1
81.4
59.6
76.8
30.9
86.4
PHDIMP [32]
50.3
62.2
19.4
77.8
68.8
90.4
41.9
97.0
56.6
68.8
11.2
98.1
53.4
64.7
23.4
82.7
54.5
67.4
20.4
85.8
P+DiMP (ixeddeeen))
41.2
49.8
18.4
66.1
45.2
12.5
87.1
62.4
21.3
90.4
47.6
58.3
18.5
77.0
61.2
78.6
PHDiMP+SpileSlleer-
B
53.7
67.1
22.2
80.2
70.1
94.5
41.8
97.2
70.7
89.2
54.2
93.9
56.1
70.3
25.7
83.7
59.2
75.3
29.1
86.8
PrDiMP.SpiikeSivvr''
55.2
70.7
24.9
79.9
71.3
95.7
45.1
97.7
72.5
91.4
59.2
95.4
57.6
73.0
27.6
83.8
60.9
78.2
32.3
87.2
TaMOs [33]
37.9
43.5
1.5
66.9
46.4
30.5
0.2
87.8
50.8
54.2
1.0
96.7
40.9
36.2
0.7
77.1
42.5
41.2
1.0
78.8
TaMOs (fuxed eveut)
44.0 57.0
3.9
90.0
46.2
43.4
46.0
1.5
78.4
48.5
77.1
TE.MOs-SpikeSSice--B
40.4 45.4
1.7
72.2
47.1
34.6
0.1
91.7
48.7
45.7
0.3
98.5
43.6
42.0
1.0
81.7
44.2
43.1
83.3
TBMos-SpllkSSllerss
41.4 48.1
1.4
71.7
48.5
36.1
0.3
95.5
45.0
18.7
0.1
98.7
40.7
29.6
0.5
80.4
43.0
36.3
0.8
82.9
TransT [27]
55.8 69.4
27.7
82.0
70.5
90.9
49.7
98.3
74.1
98.5
55.5
99.9
58.6
73.9
30.2
87.2
61.3
78.3
33.8
89.2
TransT (feed event))
51.4 67.8
11.1
80.2
28.3
89.3
41.5
57.7
57.9
12.7
59.0
78.8
Trans SpiikeSSSie B
58.0 73.4
29.4
83.8
71.9
95.9
47.7
99.3
73.4
95.7
56.9
96.2
60.5
76.6
32.0
88.4
62.1
79.9
34.6
88.7
SuusT-SSikeSSieers
59.1 75.2
30,9
84.4
72,9
97.9
52.2
99.1
76.6
99.5
65.7
99.8
60.2
76.5
32.3
87.5
63.6
82.3
37.5
90.1
Table 2: Quantative comparison on
N-Caltech DVS DVS-CIFAR10 and SL-Animals. Random and Fix denote that the input events are randomly sliced and fixed sliced, respectively. Instead, our method slices the event stream adaptively.
Method
DVS-Gesture [34]
NoCaltech10l [35]
DVS DAS-CIFARO [36]
SL-Animals [37]
Slice Type
Rondom
Fix
Ours
Random
Fix
Ours
Random
Fix
Ours
Random
Fix
Owrs
R8
93.06
93.40
94.79
77.80
73.37
79.86
78.91
77.73
81.15
85.71
83.93
88.39
ResNet-34
95.14
93.40
96.18
78.77
76.08
82.54
80.57
79.39
82.23
86.61
87.50
89,93
Swin-S
88.19
89.93
91.67
86.30
81.76
87.30
81.54
79.79
83.01
74.11
56.25
75.45
Event-based Recognition. We also conduct experiments in based recognition to evaluate the effectiveness of our proposed method. As depicted in Tab. 2, our method has a significant improvement over the fixed-sliced method, with an accuracy improvement of 2.78% and 6.46% by using ResNet-34 in DVS-Gesture and N-Caltech101, respectively. To verify that the results of our adaptive slicing method are not biased due to randomness, we add the random-slice baselines for comparison, in which the event stream is randomly sliced into event groups and fed into the ANN for training. Our method also yields better performance compared with the random-slice results.
Visualization of Adaptive Event Slicing. We visualize the tracking results to demonstrate that the dynamic slicing method is able to adapt to various motion scenarios. As shown in Fig. 5, our method obtains better tracking performance compared to fixed event inputs, i.e., the position of the prediction box is more accurate. Additionally, our dynamic event slicing method can achieve (1) edge enhancement and (2) redundancy removal to refine the event data under different tracking scenarios. However, the fixed-slice approach adopts the same slicing strategy for each event stream, leading to performance degradation.
4.3 Analysis of the Adaptive Slicing Method
Analysis of Spike Splitting Time and Event Density, To evaluate the effectiveness of our proposed method, we conducted a detailed visualization analysis, as depicted in Fig. 6, examining the rela- tionship between the locations of split points and the corresponding event stream densities. The definition of event density is detailed in the Appendix B. The analysis reveals a clear match between the positions of cuts made by SNN and the respective event density. Specifically, the SNN tends to perform more frequent cuts in regions of higher event density, while conversely, regions with lower event density experienced fewer cuts. These findings indicate that the dynamic cutting method is effectively adaptive to the varying information density within the event stream.
8

---

Table 3: Comparison of Efficiency and Speed. The comparison includes the number of operations (OPs) and tracking speed per image without image processing time.
Models
OPs (G)
Energy (m.)
Speed / img)
Performance
ANN w/o SpikeSlicer
56.36
259.26
0.045
51.0
ANN w/ SpikeSlicer
$7.09
260.11
0,060
62.4
Table 4: Experiments on different event representations with fixed or dynamic slic- ing methods. Our method yields sig- nificant improvement when using differ- event representation methods, proving SpikeSlicer's effectiveness as plug-and- play event slicer.
Table 5: Ablation studies for evaluating the proposed loss function on the SL-Animals dataset.
Slice Method Carem
Cz.1
ResNet-18
ResNet-34
Slice
83.93
SpikeSlicer
x 87.50 (43.57)
88.52
(+1.02)
SpikeSlicer
888.3 (+4.46) 89.73 (+2.23)
Table 6: Experiments with different event cell numbers N. The resulting sliced event group always has similar time interval in various N conditions.
FEI08
Time Surface
Event Spike Tensor
Vondi Grid
Slice
85.7
N/.4
81.6
78.6
SpikeSileer (ours)
59.5
86.8
$4.5
R5.6
62.4
85.9
Foent Frame
Rivent Spikk Tensor
Vourt Grid
15
25
Stice Menhed
Ascunacy
Atcunnc
Atsuway
Avg Spike Time
2.42
3.15
4.77
03.75%
SyikeSlleer (nurs)
Time Duration
16.13%
15.75%
19.08%
Analysis of Efficiency and Speed. We evaluate the performance of the ANN tracker (Trans') with or without the SpikeSlicer in terms of energy consumption and processing speed. Detailed calculations for energy consumption are in Appendix M. In Tab. 3, the SpikeSlicer stands out as a allgg-ndd-plas option. The introduction of the SpikeSlicer incurs marginal energy increase of 0.85 mJ and a small reduction in processing speed. However, these costs are offset by a significant 22.3% improvement in the RSR metric.
Evaluation of the Dynamic Hyperparameter Tuning. We use different a settings in tracking experiments to study the effect of hyperparameters. Fig. 4(b) shows that the fixed setting does not allow the spiking time step to reach the desired time step accurately, while our dynamic a tuning allows SNN to spike at the desired time step, leading better event-slicing process.
4.4 Ablation Study
Ablation Study on Event Representation Method. Since SpikeSlicer works as plug-and-play event slicing method suitable for multiple event representations, we thus conduct method through different event representation methods to further evaluate the effecctivnness. Tab. 4 demonstrates that, across various forms of event representation, the dynamic splitting method yields performance enhancements in both tracking and recognition tasks when compared to the fixed splitting method.
Ablation Study on Different Components of SPA-Loss. The results in Tab. 5 reveal that the adaptive slicing method achieves accuracy improvement, and all our proposed loss functions (including Mem and LLA) contribute to the performance, showing the effectiveness of our method.
Fren Deasky
Bosellne
dave
Currs
sirplanc
2000
4001
1000
Time Step
Baseline
SpikeSlleer (Ours)
GT
Figure 6: Visualization of spike splitting points with corresponding event density. The results of our dynamic splitting are matched real event information. The higher the event density, the SNN splits more frequently; conversely, the SNN splits more sparsely.
Figure 5: Visualization results on FE108 dataset. The white box denotes the zoom-in area. Our adap- tive event slicing method provides better tracking performance than fixed counterparts while enabling edge enhancement (a,b) and redundancy removal (c).

---

Table 7: Ablation studies for different network sizes of the SpikeSlicer. The comparison includes the model consumptions and tracking performances of PrDiMP.
Model Consumptions
Performances
Methods
Params (M)
OPs (G)
Energy (mJ)
RSR
OP.50
06.75
RPR
SpikeSlicer-Base
45.11
0.73
0.85
59.24
75.25
29.12
86.82
SpikeSticer-Small
0.42
0,56
0.69
60.88
78.19
32.34
87.19
Ablation Study on Event Cell Number. Considering that the number of event cells N may affect the SNN's 's decision on event slicing, we examine the stability of the sliced event group by varying the size of N. Tab. 6 shows that the average spike time of the SNN varies for different N, but the time duration (i.e., Avg Spike/N /N) of the resulting event groups is stable. This verifies that the SNN can effectively make cuts based on event information rather than making decisions based on the number of inputs alone. More details are provided in Appendix P.
blation Study on Different Network Sizes of the SpikeSlicer. To evaluate the performance of SpikeSlicer under different network sizes, we conduct additional ablation study (in Tab. 7) with a smaller variant, SpikeSlicer- -mall. This lightweight model contains only 0.42M parame- ters- siggiiinnfic fewer than the base model's while achieving comparable better performance across key metrics. This compact design demonstrates its potential for efficient deployment on hardware platforms, providing a strong foundation for real-world applications requiring lightweight neural networks.
5 Limitation
We summarize the limitations of this work as follows: Firstly, the SpikeSlicer process involves multi-stage SNN-ANN training, which leaves substantial room for improvement in the adaptive slicing strategy. Secondly, for recognition tasks, we convert the event stream into a single-frame representation to obtain accurate supervisory signals. This approach could be refined in the future to enable SpikeSlicer to slice stream events into multi-frame representations, which are the mainstream format. Thirdly, our experiments are conducted GPUs; however, the most suitable hardware for SNNs would be brain-inspired chips. In addition, dynamic events need to be generated and processed in real-time during inference, rather than fixed generation in advance. As result, conducting experiments on GPUs may lead to slower overall inference speeds. Extending this paradigm to brain-inspired chips with asynchronous event input is an interesting direction worth exploring the future.
6 Conclusion
In this work, we proposed SpikeSlicer, a novel event processing method that splits event streams adaptively. SpikeSlicer utilizes a spiking neural network (SNN) as event trigger, which determines the slicing time according to the generated spikes. To achieve accurate slicing, we designed the Spiking Position-aware Loss (SPA-Loss) which guides the SNN trigger spikes at the desired time step. In addition, we proposed a Feedback-Updat training strategy that allows the SNN to make accurate slicing decisions based on the ANN feedback. Extensive experiments have demonstrated the effectiveness of SpikeSlicer in yielding performance improvement in event-based object tracking and recognition tasks. In the future, we will assess SpikeSlicer's suitability for other event-based tasks, and devise more efficient training strategies for the SNN-ANN cooperative framework to optimize real-time processing in the future.
References
[1] Guillermo Gallego, Tobi Delbrück, Garrick Orchard, Chiara Bartolozzi, Brian Taba, Andrea Censi, Stefan Leutenegger, Andrew J Davison, Jörg Conradt, Kostas Danillidis, et al. Event-based vision: A survey. IEEE TPAMI, 44(1):1544 180, 2020.
[2] Jiqing Zhang, Bo Dong, Haiwei Zhang, Jianchuan Ding, Felix Heide, Baocai Yin, and Xin Yang. Spiking transformers for event- based single object tracking. In CVPR, pages 1801-8810, 2022.
[3] Jiqing Zhang, Xin Yang, Yingkai Fu, Xiaopeng Wei, Baocai Yin. and Bo Dong. Object tracking by jointly exploiting frame and event domain. In ICCV, pages 13043-13052, 2021.
[4] Jiyuan Zhang, Lulu Tang, Zhaofei Yu, Jiwen Lu, and Tiejun Huang. Spike transformer: Monocular depth estimation for spiking camera. In ECCV, pages 34-52. Springer, 2022.
10

---

[5] Yeongwoo Nam, Mohammad Mostafavi, Kuk-Jin Yoon, and Jonghyun Choi. Stereo depth from events cameras: Concentrate and focus on the future. In CVPR, pages 61 64-433,, 2022.
[6] Amos Sironi, Manuele Brambilla, Nicolas Bourdis, Xavier Lagoree, and Ryad Benosman. Hats: Histograms of averaged time surfaces for robust event-hased object classification. In CVPR, pages 1731-1740, 2018.
[7] Raymond Baldwin, Ruixu Liu, Mohammed Mutlaq Almatrafi, Vijayan K Asari, and Keigo Hirakawa. Time-ordered recent event (tore) volumes for event cameras. IEEE TPAMI, 2022.
[8] Lin Wang. Yo-Sung Ho, Kuk-Jin Yoon, et al. Event-based high dynamic range image and very high frame rate video generation using conditional generative adversarial networks. In CVPR, pages 10081-10090, 2019.
[9] Daniel Gehrig, Antonio Loquercio, Konstantinos G Derpanis, and Davide Scaramuzza. End to-end learning of representations for asynchronous event-based data. In ICCV, pages 5633 5643, 2019.
[10] Ana Maqueda, Antonio Loquercio, Guillermo Gallego, Narciso Gareía, and Davide Scaramuzza. Event- based vision meets deep learning on steering prediction for self-driving cars. In CVPR, pages 5419-5427, 2018.
[11] Lin Zhu, Xiao Wang. Yi Chang. Jianing Li, Tiejun Huang, and Yonghong Tian. Event-based video reconstruction via antential-assisted spiking neural network. In CVPR, pages 3594-3608 2022.
[12] Alex Zihao Zhu, Liangzhe Yuan, Kenneth Chaney, and Kostas Daniilidis. Unsupervised event-based learning of optical flow, depth, and egomotion. In CVPR, pages 989-997, 2019.
[13] Yansong Peng, Yueyi Zhang, Peilin Xiao, Xiaoyan Sun, and Feng Wu. Better and faster: adaptive event conversion for event-based object detection. In AAAI, volume 37, pages 2056-2064, 2023.
[14] Jianing Li, Jia Li, Lin Zhu, Xijie Xiang, Tiejun Huang, and Yonghong Tian. Asynchronous pptiiottmpporal memory network for continuous event-based object detection. JEEE TIP, 31:2975-2987, 2022.
[15] Jiqing Zhang, Yuanchen Wang, Wenxi Liu, Meng Li, Jinpeng Bai, Baocai Yin, and Xin Yang. Frame-event alignment and fusion network for high frame rate tracking. In CVPR, pages 9781-97999 2023.
[16] Inwoo Hwang. Junho Kim, and Young Min Kim. Ev-nerf: Event based neural radiance field. In WACV, pages 837 847, 2023.
[17] Viktor Rudnev, Mohamed Elgharib, Christian Theobalt, and Vladislav Golyanik. Eventnerf: Neural radiance fields from single colour event camera. In CVPR, pages 4992-5002, 2023.
[18] Ziqing Wang, Yuetong Fang, Jiahang Cao, Qiang Zhang. Zhongrui Wang, and Renjing Xu. Masked spiking transformer. In ICCV, pages 1761-1771, October 2023.
[19] Jiahang Cao, Ziqing Wang. Hanzhong Guo, Hao Cheng, Qiang Zhang, and Renjing Xu. Spiking denoising diffusion probabilistic models. WACV. pages 4912 4921, 2024.
[20] Eric Hunsberger and Chris Eliasmith. Spiking deep networks with lif neurons. arXiv preprint 110.0995,, 2015.
[21] Anthony N Burkitt. A review of the integrate andrate--nn--eee and-fire neuron model: I. homogeneous synaptic input. Biological Cybernetics, 95:1-19, 2006.
[22] Patrick Bardow, Andrew J Davison, and Stefan Leutenegger. Simultaneous optical flow and intensity estimation from an event camera. In CVPR, pages 884-892, 2016.
[23] Mike Davies, Narayan Srinivasa, Tsung- Han Lin, Gautham Chinya, Yongqiang Cao, Sri Harsha Choday, Georgios Dimou, Prasad Joshi, Nabil Imam, Shweta Jain, et Loihi: neuromorphic manycore processor with on-chip learning. IEEE Micro, 38(1):82-99 2018.
[24] Filipp Akopyan, Jun Sawada, Andrew Cassidy, Rodrigo Alvarez-lcaza, John Arthur, Paul Mcrolla, Nabil Imam, Yutaka Nakamura, Pallab Datta, Gi-Joon Nam, et al. Truenorth: Design and tool flow of a 65 mw I million neuron programmable neurosynaptic chip. IEEE TCAD, 34(10):1537-1557 2015.
[25] Qinyi Wang, Yexin Zhang, Junsong Yuan, and Yilong Lu. Space-time event clouds for gesture recognition: From rgb cameras to event cameras. In WACV, pages 1825,1835,00 2019.
[26] Zhiyu Zhu, Junhui Hou, and Xiangiang Lyu. Learning graph henh--embededd key-event back- tracing for object tracking in event clouds. NewrIPS, 35:7462 7476-777666600 2022.

---

[27] Xin Chen, Bin Yan, Jiawen Zhu, Dong Wang. Xiaoyun Yang, and Huchuan Lu. Transformer tracking. In CVPR, 2021.
[28] Goutam Bhat, Martin Danelljan, Luc Van Gool, and Radu Timofte. Learning discriminative model prediction for tracking. In ICCV, 2019.
[29] Yinda Xu, Zeyu Wang, Zuoxin Li, Ye Yuan, and Gang Yu. Siamfe++: Towards robust and accurate visual tracking with target estimation guidelines. In AAAI, 2020.
[30] Goutam Bhat, Martin Danelljan, Luc Van Gool, and Radu Timofte. Know your surroundings: Exploiting scene information for object tracking. In ECCV, 2020.
[31] Xingping Dong, Jianbing Shen, Ling Shao, and Fatih Porikli. Clnet: A compact latent network for fast adjusting siamese trackers, In ECCV, 2020.
[32] Martin Danelljan, Luc Van Gool, and Radu Timofte. Probabilistic regression for visual tracking. In CVPR, 2020.
[33] Christoph Mayer, Martin Danelljan, Ming- Hsuan Yang. Vittorio Ferrari, Luc Van Gool, and Alina Kuznetsova. Beyond sot: Tracking multiple generic objects at once. In WACV, pages 6826-6836. 2024.
[34] Arnon Amir, Brian Taba, David Berg, Timothy Melano, Jeffrey McKinstry, Carmelo Nolfo, Tapan Nayak, Alexander Andreopoulos, Guillaume Garreau, Marcela Mendoza, et al. A low power, fully event-based gesture recognition system. In CVPR, pages 7243-7252. 2017.
[35] Garrick Orchard, Ajinkya Jayawant, Gregory K Cohen, and Nitish Thakor. Converting static image datasets to spiking neuromorphic datasets using saccades. Frontiers Neuroscience, 9:437, 2015.
[36] Hongmin Li, Hanchao Liu, Xiangyang Ji, Guoqi Li, and Luping Shi. Cifar10-dvs: an event-stream dataset for object classification. Frontiers in Neuroscience, 11:244131, 2017.
[37] Ajay Vasudevan, Pablo Negri, Camila Di lelsi, Bernabe Linares- Barranco, and Teresa Sl-animals-dvss event went-driven sign language animals dataset. Pattern Analysis and Applications, pages 1-16, 2022.
[38] Arjun Roy, Manish Nagaraj. Chamika Mihiranga Liyanagedera, and Kaushik Roy. Live demonstration: Real-time event-based speed detection using spiking neural networks. In CVPR, pages 4080-4081, 2023.
[39] Alberto Viale, Alberto Marchisio, Maurizio Martina, Guido Masera, and Muhammad Shafique. Carsnn: An efficient spiking neural network for event-based autonomous cars on the loihi neuromorphic research processor. In IJCNN, pages 1-10. IEEE, 2021.
[40] Fangwen Yu, Yujie Wu, Songehen Ma, Mingkun Xu, Hongyi Li, Huanyu Qu, Chenhang Song, Taoyi Wang. Rong Zhao, and Luping Shi. Brain-inspired multimodal hybrid neural network for robot place recognition. Science Robotics, 8(78)eabm6996, 2023.
[41] Jesse Hagenaars, Federico Paredes-Vallés, and Guido De Croon. Self- superviee learning of event event-based optical flow with spiking neural networks. NewrIPS, 34:7167- -179, 2021.
[42] Man Yao, Huanhuan Gao, Guangshe Zhao, Dingheng Wang. Yihan Lin, Zhaoxu Yang, and Guoqi Li. Temporal- attention spiking neural networks for event streams classification. In ICCV, pages 10221 10230, 2021.
[43] Xavier Lagorce, Garrick Orchard, Francesco Galluppi, Bertram Shi, and Ryad Benosman. Hots: a hierarchy event-based time-surfaces for pattern recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(7):1346 1359,,159,0000 2016.
[44] Priyadarshini Panda, Sai Aparn Aketi, and Kaushik Roy. Toward scalable, efficient, and accurate deep spiking neural networks with backward residual connections. stochastic softmax, and hybridization. Frontiers in Neuroscience, 14:653, 2020.
[45] Man Yao, Guangshe Zhao, Hengyu Zhang, Yifan Hu, Lei Deng, Yonghong Tian, Bo Xu, and Guoqi Li. Attention spiking neural networks. IEEE TPAMI, 2023.
[46] Ze Liu, Yutong Lin, Yue Cao, Han Hu. Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In ICCV, pages 1002-10022, 2021.
12

---

[47] Alexey Dosovitskiy. Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai. Thomas Unterthiner, Mostafa Dehghami, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition scale. In ICLR, 2020.
[48] Junho Kim. Jachyeok Bae, Gangin Park, Dongsu Zhang, and Young Min Kim. N-imagenet: Towards robust, fine-grained object recognition with event cameras, In ICCV, pages 2146-2156. 2021.
13

---

Appendix
A Details of our Motivations
To clarify the motivation behind our dynamic event stream slicing algorithm, this section tells the details.
A.1 Motivation for Proposing Dynamic Event Stream Slicing Algorithm
The process of event-to- tonnttttrrreeentttionn conversion mainly divided into two steps: Step 1. Slice the raw event stream into multiple sub-event stream, and Step 2. convert these sub-event streams into representations using various event representation methods. While much work has focused on optimizing event representation (Step 2) to extract better event information, including time surface and EST, they do not address the issues arised with fixed slicing (e.g., resulting non-uniform event in scenarios with changing motion speed). Despite event slicing being a small part of the overall pipeline, it is a critical point. This because the event stream very sensitive slicing, and the model performance fluctuates very much for different slicing methods, as proved by extensive experiments in Appendix C.
To better address this issue, we introduced the dynamic slicing method SpikeSlicer. Meanwhile, SpikeSlicer is guided by downstream task feedback to ensure that the new sub-streams could enhance downstream task performance.
A.2 Motivation for for Using SNN as a Slicing Trigger
The reason why we choose SNN as the event slicing trigger is twofold:
Utilizing SNNs on neuromorphic hardware for processing event streams is low-energy and low-latency [23, 24].
Deployed on neuromorphic hardware, SNNs can process event streams asynchronously [38- 40], conserving energy when there is no data input-a capability that GPUs, operating synchronously, lack.
Due to the aforementioned reasons, there is considerable amount of research [41, 11] employing Spiking Neural Networks (SNNs) for event data. Although these SNNs simulated on GPU platforms, the models resulting from such simulations could be deployed on neuromorphic hardware.
A.3 Contribution for Using SNN as Slicing Trigger
We propose a new cooperative paradigm where SNN acts as an efficient, low-energy data processor to assist the ANN in improving downstream performance. This is brand-new SNN-ANN cooperation way, paving the way for future event-related implementation on neuromorphic chips.
B Definition of Event Density
In our experiments, we investigated the relationship between the location of the split point determined by the SNN and the density of the corresponding event stream. The event density, denoted as ((t), measures the concentration of events in given event stream over time. It mathematically defined as the rate at which events occur per timestep, expressed as a function of:
6N
D
(11)
8t
where Dtt) the event density timestep t, 8.V is the number of events occurring within a small time interval öt around t. This definition enables precise quantification of event concentration, offering insights into the temporal distribution of events at any given moment. Our empirical analysis shows that there is significant correlation between the split points of the SNN and the event density, Notably, regions higher event density, the SNN exhibits a tendency to perform more frequent split. In contrast, in regions with lower event density, the number of split is lower. This behavior emphasizes the adaptability of our proposed dynamic slicing method, SpikeSlicer, to the fluctuating information density in the event stream.
14

---

C Sensitivity Analysis of Fixed Event Slicing Method
To demonstrate that events are sensitive slicing by fixed methods, and emphasize the importance of proposing dynamic event slicing approach, we have conducted a total of 60 experiments with different models to investigate the impact of different slicing techniques and different numbers of slices on the performance in downstream tasks.
In our experiment, we employed two fixed slicing methods: (1). Slicing with a fixed number of events, and (2). Slicing with fixed duration. N denotes the number of resulting event slices. Experimental results are detailed as follows in Tab. and Fig.
Table 8: The sensitivity analysis of fixed event slicing on N-Caltech The results demonstrate that the event sensitive to the fixed slicing method (slicing by fixed time or event count), thereby affirming the need for proposing dynamic slicing method.
N-Caltech101
N
8
24
26
28
30
Mean
Var
ResNet18
Fixed Cown
70.96
75.26
75.39
75.30
76.09
73.95
74.09
73.80
76.40
75.3
75.45
73.60
71.94
71.01
71.17
73.98
ResNet18
Fixed Time
62.90
72.64
76.38
74.48
74.91
73.70
74.30
74.69
76.95
74.75
74.46
74.42
71.61
71.52
69.69
73.16
10.80
ResNet34
Fixed Count
79.55
72.19
76.98
78.22
77.14
77.40
76.78
76.90
78.14
77.06
76.91
74.85
74.76
76.91
73.07
76.19
2.90
ResNet34
Pixed Time
75.92 65.
78.29
78.20
78.48
76.23
77.76
76.57
75.94
76.80
76.61
75.91
75.11
74.76
74.19
75.74
9.15
0.78
0.76
Accuracy (%)
0.74
0.72
0.70
ResNet18-Fixed Count
ResNet 18-Fixed Time
ResNet34-Fiie
0.64
ResNe:34-Fioo Time
10
15
25
30
Number of Sliced Event
Figure 7: Visualization of sensitivity amalysis on N-Caltech100 101 dataset. The fluctuations in accuracy for different numbers of sliced event with different fixed slicing methods are significant, demonstrating that events are very sensitive to fixed slicing methods.
The results indicate significant fluctuations (large variance) in downstream performance based on the slicing method and the number of slices used. We believe this addition effectively demonstrates the sensitivity of event streams to slicing techniques, confirming the need for motivation to propose dynamic slicing of event streams. Additionally, the accuracy achieved using the dynamic slicing method (82.54% ResNet34) surpasses that of any fixed slicing approach (with the highest being 78.48% 78.48%) further substantiating the efficacy of the dynamic method in our study.
D Illustration of SpikeSlicer vs. Fixed Slicing Method
In order to more intuitively show the difference between our dynamic event slicing method and the traditional fixed slicing method, specifically illustrate these methods through Fig. 1.
Fig. 1(a) denotes that each resulting sliced event has the same duration, and Fig. 1(b) denotes that the number of event points contained in each resulting sliced event is the same. Since event stream are usually unevenly distributed, fixed cutting method often leads non-uniform event information (e.g., in scenarios with changing motion speed). In contrast, our approach decides the optimal slice position through feedback from downstream ANN by using an SNN as the event slicing trigger.
E Difference between Event Slicing and Event Representation
worth noting that our work focuses on the slicing the event stream rather than focusing on event representation. Event representation refers to the process of event information extraction
15

---

Table 9: Experiments on different event representations with fixed (including fixed time and fixed event count) or dynamic slicing methods. Our SpikeSlicer yields significant improvement when using different event representation methods.
DVSGesture
Event Frame Event Spike Tensor Voxel Grid
Fix Duration
93.75%
93.75%
88.54%
Fix Event Count 93.06%
94.79%
88%
SpikeSlicer (ours) 94.79 %
95.49 %
89.24%
that is performed after the event stream has been sliced into sub-event stream, and the resulting event representation meets the neural network input requirements. Thus, our dynamic slicing process and event representation can be used at the same time, either better slicing or representation method benefits the feature extraction with neural network, thus improving performance.
To validate the effectiveness of our slicing approach, we supplement the event-based recognition task below. We compare the downstream performance of three different event representation methods (including Event Frame [10], Event Spike Tensor (EST [9]) and Voxel Grid [12]) on the DVSGesture dataset under fixed slicing and dynamic slicing method in Tab. 9.
F Definition of Raw Event Group
Based on the definition of event field from [9], we here define general version of raw event group representation as a mapping M E + T between the set & and a tensor T:
Definition 1 (Raw event group). Based on measurement Lcondition. raw event group are grid-like tensors defined in continuous space and time:
G+ (z, y, condition)
Σ oxEE+
Leondition (2, v,t IK, yx)) tx),,
(12)
where + denotes the event polarity; lcondition sets the specific approach of representation to each event, e.g., condition = (E = -M} denotes the number of event points in each grid-like tensor is fixed to M, i.e., slicing event stream E by event count; or condition = {At = tz} denotes the time interval of event points in each grid-like tensor is fixed to tz. G+ are grid tensors with x E {0, I,..., W 1}. y E {0, I..., H - 11. and E {to, to + Atzl...,, to + Btt, ). where to is the first time stamp, Atzi is the bin size determined by the splitting condition, and B is the number of temporal bins. Eq. (12) converts raw event into grid-like raw event group by a Dirac pulse [9] in the space-tima time manifold. The resulting G+ gives continuous time representation of & which preserves the event's ' information.
However, if such raw event groups are then converted into event representation (e.g. voxel grid [22], time surface [43]), the generated event representation imprecise due to the fact that the process of 1 condition is fixed, leading to both spatial and temporal information loss. The main objective of this study is to solve the problem of fixed slicing of the event stream and to provide dynamic segmentation scheme.
G Reason for Using No-reset Membrane Potential
We first recall the definition of original membrane potential Vin]:
[nn] 3V [n= 1] >/[]]
(13)
Sin] @(V [n] vl))
(14)
[nn] V|n[(1- - S((n) + Vreset>
(15)
where the the neuron will reset its membrane potential to Vreset 0s.. at time once it trigger spike Sin]. As described in Sec. 3.3, we choose guide the membrane potential without reset stage
(Eq. 15) [[]] (or named no-reset membrane potential), instead of the normal membrane potential
V [n]]. The no-reset membrane potential is defined similarly as [in]:
Unn] 3U(n In - 11+ 7/[],
(16)
S[n] = (U[[x) 0.)))
(17)
16

---

but the neuron does not reset its membrane potential in this condition. The reason behind this choice that the reset process will affect the guidance of the Vnn]. Specifically, suppose we expect the neuron to fire spike at + 1, but if spike just occurs at time n. the membrane potential Vnn] will reset to Vreset, consequently leading to small value for Vln + 1]. Based on the SPA-Loss function, V[n + would then be guided by large expected membrane potential value (above the threshold). However, this would incorrectly guide the membrane potential after resetting to the desired membrane potential, rather than guiding the true membrane potential as intended. Therefore, we choose to use the no-reset membrane potential Unr] to effectively guide the spiking neuron to fire spike at the specified location.
H Proof of Proposition
Proposition 1. Suppose the input event cell sequence has length N, desired spiking time is n² (n' E (0, 10,,,..., N.), the membrane potential at time n" satisfying the constraints:
Vn U'nn) S max( + + Vt.),
(18)
where T[=*) is the input synaptic current from Eq. Then the spiking neuron fires a spike at time n and does not excite spikes at neighboring moments.
Proof. Here we consider two conditions that affect the spiking state at moment n²:
(I) Membrane potential at time n* is too small to emit a spike.
(2) Membrane potential at time n² is too large, affecting neighboring moment spiking states.
To satisfy the condition (1), we only need to guide the membrane potential U[n"] reach the threshold VzA at time n", thus the upper bound of U [[n)) V4.- In condition (2), we need to consider
the state of the membrane potential n² and n + 1. We first exhibit the accumulation rules of membrane potential:
(19) However, if U [n]]] is too large, this may cause the membrane potential U(n" - 1] to exceed the threshold and occur spike generation prematurely, and then the membrane potential will immediately drop to reset value (Eq. 15). This will leave the membrane potential U [n[n]] at very low value, making it difficult to trigger a spike. Hence, we should control the membrane potential not to exceed the threshold value at moment n
U|n'] = 3U - 1] >/[n]
Unn" 1] (U[n") T[n")) S Vi
(20)
= Unn"] s 3nV + TI[[n=]
(21)
Thus, the upper bound of U(n)] = Ve, + [[[]]. However, the leaky factor 3 is small, there exists possibility that Ve. + 7 [mn)]] ] s Ven, thus we set the upper bound of U [n*]] as max(8Va. T[nn), V..).
Next, we consider whether the spike at time n² affects the pulse state at time n' + 1 Since the neuron at the n* moment has already completed the spike generation before accumulating U[n" + 1]. Therefore the membrane potential at n² does not affect the neuronal state at n + 1. In sum, if the membrane potential satisfies: Via I U[")] I max(3Vin + >[[n"], Voa), ), the spiking neuron fires a spike at moment n² and does not excite spikes at neighboring moments.
I More Explanations in Spiking Position-aware Loss
L.1 Details in Membrane Potential-driven Loss
In Sec. 3, we explore the range the expected membrane potential U[rr'] and ensure its rationality by proposition (proved in Appendix. H). To understand the setting of the membrane potential more easily, we visualize the boundary cases in Fig. 8.
The lower bound case means that the membrane potential at the desired index should be at least Va to activate a spike, and the upper bound case guides the membrane potential to exceed the threshold but prevents generating spikes in the previous time step. Hence, the desired membrane potential should be bounded in Vin, max (5Vr. 7[[n), Vin)] and a E [0, 1] (Eq. 8) balances the desired membrane potential U [2"' between Ulower and U uppper-
17

---

Desired U[n*]
(1 a) Unevr Upper
Supervised Men.
Va
Unpper
Unver
11-2
11-1
n+1
Time Step
Figure 8: Visualization of the boundary cases when controlling the desired membrane potential, where the 'heart-like' point denotes the lower bound case and the 'moon-like' point denotes the upper bound case.
1.2 Details in Linear-assuming loss
As described in Sec. 3.3.2, suppose we expect the SNN to trigger a spike at a later time, if there exists hill effect, the earlier membrane potential always reaches the excited state sooner and turns the neuron into the resting state to suppress the spiking generation at later moments. To address this challenge, we expect the later membrane potential to satisfy: (1) the later membrane potential should be larger than the current membrane potential to reverse the hill effect, and (2) the later membrane potential should exceed the threshold to fire a spike. Hence, we assume that the membrane potential in this condition should increase monotonically with the time step, as illustrated in Fig. 9.
Linear Increasing Assumption
Supervised Mem.
Va
Vin
Blc 11
Vix
n
Time Step
Figure 9: Visualization of our expected linearly increasing membrane potential.
Then we use the LA-Loss to supervise the membrane potential at ne to reach "s Vin in order for the latter membrane potential at the n" to reach Vzh, satisfying both (1) and (2).
1.3 Details of Dynamic Hyperparameter Tuning
To deeply explore the control of the hyperparameter a,, we first analyze the effect in Eq. 8. When reaches its maximum value (i.e.a 1), the desired membrane potential evolves to Unppers which corresponds to a situation where the membrane potential just reaches the threshold at the previous moment (in Fig. 8). That is, a increases, the previous moment is more likely to generate spike, driving the spiking time earlier. Recalling observation 2, taking a large-a scenario as an example, the SNN fails spike a later time step due the alpha being too large, limiting the neuron's ability to spike later. Hence, we expect the a to decrease to allow the desired index to decrease as well; similarly for small a scenarios. To summarize, we hope the ar is updated in the same direction the desired index, we update a by setting:
N n.,/NN111"
a.grad =
Σ
(n
(22)
N. (n
a a 11--22 a-2 Σ
n
(23)
18

---

where grad denotes the gradient of a and denotes derivative operation. The explanations of other math symbols can be found in the main text and Alg.
Visualization of the SNN Training Process
To validate whether the proposed feedback-update strategy can serve a guiding role during the initial stages of training, we have visualized the training process of the SNN and presented it in Figure 1 of the supplementary PDF rebuttal file. As anticipated, the training of the SNN exhibited fluctuations during the initial training stage, which might be attributed to the instability of the event quality obtained from dynamic slicing at this early phase. However, as training progressed, the loss of the SNN gradually stabilized and decreased, converging towards desired outcome. Correspondingll, the slicing times progressively converged towards the desired spiking index. Therefore, although initial exploration may require several steps, our proposed training method is capable of offering effective guidance.
SNN Loss Over Time
Slice Index Over Time
stagr
SpikeSlivarsl
0.20
2
SpikeSliesr3
SPA-Loos
0.15
SpikeSlizer 4
Spike Index
Warm-up Stage
Guidunce Stage
0.10
desired spiking time
SpikeSliorr-]
Spike8lieen22
0.05
SpikeSliees-3
Wir Stage
Guidance
SpikeSliorr4
Training Step (a)
Training Step
(b)
Figure 10: Visualization of the SNN Training Process. As shown in the gray box in (a), the SNN loss fluctuates considerably during the initial training stage. This is due to the instability in the quality of dynamic sliced events during the early phase (warm-up stage). However, as training progresses, the SNN loss gradually stabilizes and converges (guidance stage), with the corresponding slicing time (b) also converging towards the desired spike index.
K Implementation Details in Toy Experiments
To validate the effectiveness of SPA-Loss, we set up the toy task:
Input N randomized event cells, expect the SNN to slice a specified time step n² and there exists certain probability of interfering with SNN slice at other time steps.
In detail, we set the time step within the range [1, 30) and the max number of iterations as 800. We utilize a lightweight convolutional SNN with random initialization. We compare our proposed SPA-Loss function with common mean square error (MSE-Loss) and cross-entropy loss (CE-Loss). The results in Fig. demonstrate that our proposed SPA-Loss can guide the SNN to spike at the desired timestep accurately with fewer convergence iterations than standard MSE-Loss and CE-Loss, paving the way for complex event-based vision tasks in Sec. 4.2. More experiments on the beginner's arena are shown in Appendix N.
L Implementation Details in Event-based Task
We adopt a spiking neural network with structure: PP-AAdP2-LNN-F-LMM
which consists of three convolutional layers and two linear layers, no residual block or attention are used. [i]C(j] denotes a convolutional layer with the output channel i and the kernel size j; GN denotes group normalization; AvgP/k/ and AdaP(k) mean the average pooling and adaptive pooling with kernel size k; LN denotes the linear layer. We choose the neuron as the activation function. We adopt the SGD optimizer and set the initial learning rate as le-4, along with the cosine learning rate scheduler. SNN models are trained with batch size 32, Each experiment is conducted in an NVIDIA 4090 GPU.
19

---

L.1 Event-based Object Tracking
Datasets. The FE108 dataset [3] is an extensive event-based dataset for single object tracking, including 21 different object classes and several challenging scenes, e.g., low-light (LL.) and high dynamic range (HDR). The event streams equips a 346x260 pixels dynamic vision sensor (DVS). We choose 54 sequences for training ANNs, 22 sequences for training SNNs and the rest 32 sequences for testing.
captured by DAVIS346 event-based camera, which
Evaluation Metries. To show the quantitative performance of each tracker, we utilize three widely used metrics: success rate (Suc.), precision rate (Prec). normed precision rate (N-Prec), and overlap precision (OP). These metrics represent the percentage of three particular types of frames. Success rate is the frame of that overlap between the ground truth and the predicted bounding box is larger than threshold; Precision rate focuses on the frame of the center distance between ground truth and predicted bounding box within a given threshold: OP represents SR with thres as the threshold. We employ the area under curve (AUC) to represent the success rate. The precision score associated with 20 30-pixel threshold.
Label Settings. In this paper, since the original event dataset only provided labels fixed frame rates, we employed a linear interpolation method to obtain corresponding labels for each more refined event cell. For example, suppose that in the original event dataset, sub-event stream E with period of T (i.e., / E [1, 12)) has labels (le (ey) corresponding to moments 4,, l2, respectively. If the number of event cells in this interval is N, then each event cell represents the event with the T time range of {[tı, t1 + 1 [ + N. 11 + f...... and the label for each interval can be derived through linear interpolation using [4,,44 } and the number of event cells N. Thus, predictions at any slicing interval have corresponding labels for supervised learning. To ensure faireess, all tracking experiments in this paper utilize the aforementioned method to process the event dataset.
L.2 Event-based Recognition
[34] dataset contains 11 hand gestures from 29 subjects under illumination conditions, recorded by DVS128.
DVS-Gesture. The
N-Caltech101. The N-Caltech101 dataset [35] incorporates 8,831 event- event-based images, with 180x240 resolution and 101 classes, generated from the original Caltech101 dataset through an event-based sensor.
The DVS-CIFARI dataset [36] is an event-stream dataset designed for object classification. It consists of 10,000 event streams, created by converting the frame-basee images from the CIFAR-10 10 dataset using an event-based sensor with a resolution of 128x128 pixels. This dataset presents an intermediate level of difficulty, featuring 10 distinct classes.
DVS-CIFARIO.
SL-Animals. The SL-Animal database [37] features DVS recordings of individuals performing sign language gestures representing various animals, captured as a continuous spike flow with very low latency. This dataset includes approximately 1 1100 samples from 58 subjects, each performing 19 different sign language gestures in isolation across various scenarios, offering challenging evaluation platform for this emerging technology.
M Theoretical Energy Consumption Calculation
To calculate the theoretical energy consumption, we begin by determining the synaptic operations (SOPs). The SOPs for each block in the SNN can be calculated using the following equation:
SOPs(1) = fr x T FLOPs(!)
(24)
where denotes the block number in the SNN. fr is the firing rate of the input spike train of the block and T is the time step of the spike neuron. FLOPs([) refers to floating point operations of / block, (MAC) operations. And SOPs are the number of
which is the number of haltiply--nd-accuumul
baske accumulate (AC) operations.
To estimate the theoretical energy consumption of SNN, we assume that the MAC and AC operations are implemented on 45mm hardware, with energy costs of FMAC = 4.6pJJ and FAG = 0.9pJ, respectively. According to [44, 45), the calculation for the theoretical energy consumption of SNN is
20

---

given by:
Diffiusion = EMAC x FLOP
SNN
N
M
(25)
EAC x
Σ
SOP
SNN
SOP
SNNF
m=
m=1
where N and M represent the total number of layers of Conv and FC, EMAC and EAC represent the energy cost of MAC and AC operation, FLOPP SNNC denotes the FLOOSs of the first Conv layer, SOP SNNConv and SNN are the SOPs of th Conv and m FC layer, respectively.
N
More Experiments on Beginner's Arena
Problem Setup. To verify the accuracy of our proposed slicing method, we expect SNN can slice events at the specified time step. We set up two scenarios and only show the difficult task (II) in the main text:
Task (I): Input T identical event cells, expect the SNN to slice at a specified time step T*
Task (II): Input T randomized event cells, expect the SNN slice at specified time step T*, but there exists a certain probability of interfering with SNN to slice al other locations.
Task (I) aims to verify whether our proposed slicing strategy can accurately locate the optimal point; To the robustness of our method, task (II) simulates complex event stream processing with random inputs and adds random noise to affect the SNN with wrong labels.
Table 10: Results on simple event slicing tasks with SPA-Loss.
Input Size
Time Steps
Parameter
Iterations to Convergence J
32x 32
30
0.52M
75
Task (I)
64x
30
2.02M
32x
100
0.52M
29
Task (II)
64x
100
2.02M
88
We adopt a lightweight SNN (0.25M/2.02M) for our experiments. T is randomly selected within range [0, 7j. The experimental results presented are the average of the results obtained by setting up three random seeds. As shown in Tab. 10, SNN requires only small number iterations to converge to the specified slicing time based on the SPA-Loss. For the complex task with random inputs and disturbances in task (II), SNN can still converge fast and even faster find the specified cut point compared with task(I). This simple experiment demonstrates that SPA-Loss effectively supervise the SNN pulsing at the specified location, which paves the way for experiments on adaptive event slicing in real scenarios.
Statistics of Dynamic Slicing (SpikeSlicer) vs. Fixed Slicing
In this section, compare the statistics results of the resulting events sliced by different slicing methods.
Symbol Description:
the total event stream E; the resulting sliced sub-event stream list by SNN:
Ernes E4. Fb.. F the resulting sliced sub-event stream list by fixed slicing method: Efintime
[E E
In the tracking task, the average duration of each sub-event stream E. [1) is 65ms (cor- responding to 13 event cells, and the duration the event stream contained in each event cell is 5ms). The maximum duration of each sub-event stream is 100ms, and the minimum duration is 30ms, while for our comparison of the
approach, the duration of each sub-event stream E (j [1. M,]) is fixed at 75ms. The following Tab. O and Fig. 11 show the specific statistics of adaptive slicing vs. fixed slicing:
P Statistics of Resulting Slicing Duration
To further verify the stability and effectiveness of the dynamic slieing method, we explore the results of our method by changing the number of event cells in the event recognition task. Neell indicates
21

---

Method
Avg Cell Num Cell AvmlAvg Duration Min Duration 25th Duration 75th Duration Max Duration
SpikeSlicer
12.99
3,96
65ms
25ms
50ms
80ms
100ms
Slice by fixed duration
75ms
Table 11: Statistic results of dynamic slicing method (our SpikeSlicer) and fixed slicing method.
Sliced Event Distribution
140
Fixed Slice
Adaptive Slice (ours)
120
100
Frequency
80
60
40
20
30 40
50
60
70
80
90
100
Duration of Sliced Event (ms)
Figure 11: Visualization of sliced event distribution. Our method can return the sliced events with different durations, while the fixed method can only generate sliced events with fixed durations.
that event stream is divided into Neell event cells, and the larger the Neell implies that the event stream is divided into more fine-gras grained event cell sequences that are capable of better represent the raw event stream (as mentioned in Sec. 3.1).
Calculation Process: Suppose the whole event stream (duration = T is divided into 15 event cells, if the SNN is trained to sliced the event with 2.42 (average) event cells, which means that the sliced sub-event stream E% contains event data which lasts duration of T5 * 2.42 * 16.13%T.
Table 12:
Experiments of SpikeSlicer with different event cell numbers N. The resulting sliced event group always has similar time interval in various N conditions.
Nestt Avg Cell Num
15
20
2.42
3.15
4.77
Percentage Duration
13%
15.75%
19.08%
The experimental results show that the percentage of the duration of each sub-event stream to the total event stream duration after the adaptive slicing is relatively stable, i.e., the fineness of the event cell does affect the event information contained in each sub-event stream after the slicing process, which proves the robustness and effectiveness of the dynamic slicing process of our method. We also illustrate the event percentage change during training in Fig, 12.
More Experiments with Latest Models
To enhance the credibility and robustness of our results, we have incorporated state-of the-th---arrrr- art models: Swin Transformer (Swin'T [46]) and Vision Transformer (VTT [47]), further validate the efficacy of our algorithm in event-based recognition tasks (Tab. 13):
Experiment Settings: We choose SwinT-small and ViT-small for comparisons on the DVSGesture dataset. Other settings consistent with the main experiments.
22

---

N=10
N=15
N-20
Event Peromiage (%)
0.4
N-25
N-30
0.3
0.2
0.1
20
100
Epoch
Figure 12: Visualization of the percentage of event in various Neell conditions during training. SpikeSlicer can always split and obtain sub-event groups with similar time intervals.
Table 13: Experiments of utilizing SpikeSlicer latest recognition backbones.
Method
Random Slice Fixed Slice
Ours
SwinTT [46]
88. 19
89.93
91.67(+1.74%%
VIT [47] 87.50
85.07
88.54(+3.47%)
Results demonstrate that the SpikeSlicer also yields performance improvement in recognition tasks with the latest backbones.
R
More Experiments with Complex Neuromorphic Dataset
To further validate the proposed approach, we conduct experiments on more complex neuromorphic N-ImageNet [48]:
Table 14: Experiments utilizing SpikeSlicer on complex dataset N-ImageNet [48]
Slice Method Random Slice Fixed Slice
Ours
Re
40.98
39.43
45.48(+6.05%%
S
Impact Statement
This paper proposes effective event processing method and also provides novel SNN-ANN cooperation paradigm, aiming to inspire further research and development in energy-efficien and high-performance computing. We do not anticipate direct negative impact from our work.
T Summary
To sum up. SpikeSlicer is designed plug-and-plat algorithm for dynamic event stream slicing. benchmarked against baselines that employ fixed event stream slicing methods, proving the effectiveness of our method. In addition, approach is versatile and can be applied in any event- based vision task, not limited to recognition or single object tracking scenarios.
Notably, SpikeSlicer also provides brand-new SNN-ANN cooperation paradigm, where the SNN acts as an efficient, low-energy data processor to assist the ANN improving downstream performance, injecting new perspectives and potential avenues of exploration.
23

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility. transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], , [No] or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include it (after eventual revisions) with the final version your paper, and its final version will published with the paper.
The reviewers of your paper will be asked use the checklist as one of the factors their evaluation. While "[Yes] is generally preferable to "[No] ", it is perfectly acceptable to answer "[No] provvdddd proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] or "[NA]" is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer is often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS paper checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction of this paper clearly reflect the contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
24

---

Answer: [Yes]
Justification: Please see Discussion and Conclusion.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
model well-specificatie
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-ted system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification:
See Proposition and its proof in Appendix H.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
25

---

Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We have provided the experiment details in Appendix L..
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The experiment details are provided in the paper. We will release the code soon.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (kttps://nips.ce cc/ for more details.
public/guie htglll//guudee//oddeuummssssssoo
26

---

While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https:
/ http://ulllttuuuuudede/o//eeaaaa http///ooSSuuiiisseeeeee ieiiionooolyly for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
The experiment details are provided in the Appendix L, where the network architecture, optimizer, batch number and other information are introduced.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification:
We fixed the random seed.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
27

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of emrors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Each experiment is conducted with NVIDIA 4090 GPU, More details provided in Appendix L.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't 't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: http///wwr....ooololcl///ttttear
Answer: [Yes]
Justification:
The paper is with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations their jurisdiction..
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We provide the impact statement in Appendix
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
28

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deploymeet of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification:
This paper poses such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We cited the used assets correctly.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not introduce any new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mooel part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
This paper does involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing research with human subjects.
30

---

Guidelines:
The answer NA means that the paper does involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
31


Reviews:
[{"review_id": "0izgpI1PkS", "reviewer": "Reviewer_ij7X", "summary": "The authors designed a plug-and-play event processing method, SpikeSlicer, to split event streams with an adaptive amount. The proposed method is a lightweight SNN, constrained by a custom Spiking Position-aware Loss (SPA-Loss) to regulate neuron states. Additionally, a downstream ANN refines the slicing decisions using a feedback-update training strategy.", "questions": "The input to SNNs also involves a time constant definition, which implicitly includes the concept of a time window. How did the authors determine this time constant?", "limitations": "There is not much discussion on this aspect.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The proposed plug-and-play event processing method, SpikeSlicer, based on SNN representation, which can be used for various vision tasks with event cameras. \n* The downstream ANN refines the slicing decisions using a feedback-update training strategy, and the performance of the downstream ANN provides feedback to adjust the representation.\n* Experimental results demonstrate that SpikeSlicer can effectively enhance the performance of object tracking and recognition with event cameras, while also leveraging the advantages of neural computation in processing speed and power consumption.", "weaknesses": "* The comparison algorithm for event-based object tracking, DiMP, is from 2019. Why not try the latest methods? In recent years, many studies have focused on improving the effectiveness of event stream representation to enhance the performance of event vision tasks. \n* Many methods for object detection and tracking with event cameras have not been compared.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1:** *The comparison algorithm for event-based object tracking, DiMP, is from 2019. Why not try the latest methods? In recent years, many studies have focused on improving the effectiveness of event stream representation to enhance the performance of event vision tasks; Many methods for object detection and tracking with event cameras have not been compared.*\n\n**A1:**\nThank you for your suggestion! Due to time and resource constraints, we have incorporated a recent tracking model, TaMOs (2024) [1], and tested it on the FE108 low light dataset. The results are displayed in the table below, from which it can be observed that the tracking performances under the SpikeSlicer gain improvement. Consequently, this substantiates the efficacy of our method. \n\n\n|       |                  |                     |                 |                  | \n|---------------|:------:|:----:|:----:|:----:|\n|   **Methods**            | RSR  | OP$_{0.50}$ |  RPR  | Norm RPR  | \n| TaMOs | 46.35 | 30.53          | 87.75\t          | 62.59  | \n| TaMOs **(+SpikeSlicer)** | **46.56** | **31.20**         |  86.17 |\t**65.54** |\n\n---\n\n**Q2:** *Many methods for object detection and tracking with event cameras have not been compared.*\n\n**A2:**\nThank you for your comment! Regarding the experiments on object detection, we collected a multi-object event-based dataset and tested it using multiple baselines. The experimental results are presented in the rebuttal PDF document, with the specific experimental setups outlined in the response to reviewer HdTR. These results demonstrate the performance improvements in real-world object detection scenarios.\nWe will include these results in a future version of our manuscript. Thanks!\n\n\n\n---\n\n**Q3:** *The input to SNNs also involves a time constant definition, which implicitly includes the concept of a time window. How did the authors determine this time constant?*\n\n**A3:**\nThank you for your question! The input of the SNN is $N$ event cells, so the duration of each event cell (time window) is given by $\\delta t = T/N$. We do not set the time window directly; instead, we specify the number $N$, and the time window is implicitly determined during the generation of the event cells.\n\n\n---\n\n***Reference:***\n\n[1] Mayer C, Danelljan M, Yang M H, et al. Beyond SOT: Tracking Multiple Generic Objects at Once. WACV, 2024.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XXYKcVIzrI", "reviewer": "Reviewer_HdTR", "summary": "This work proposes a novel method for adaptively sample event data and subsequently preprocess it, utilizing a spiking neural networks (SNNs) as module.\n\nThe sampling method involves a feedback mechanism that triggers the activation of the SNN.", "questions": "no questions", "limitations": "The implementation of the code appears to be challenging, which may affect its reproducibility.\nThe application of this algorithm in embedded systems seems to be constrained due to the use of Spiking Neural Networks (SNNs).\nFurthermore, it is unclear whether the code will be made publicly available.", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "Tests are done on dataset with different lighting conditions. Being robust to different event rates.", "weaknesses": "The experiments conducted do not contain tasks such as optical flow, object detection, or image reconstruction. The type of tasks tested is limited.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1:** *The experiments conducted do not contain tasks such as optical flow, object detection, or image reconstruction. The type of tasks tested is limited.*\n\n**A1:**\nThank you for your suggestion. Due to limited time and resources, we have endeavored to incorporate a variety of task types. Specifically, we have added **four** tasks: (1). Lips Reading. (2). Human Gait Recognition. (3). Camera Pose Relocalization. (4). Object Detection in Real-world Environment. Details of all experimental setups and results are as follows:\n\n\n### **(1) Lips Reading**\n\n*Introduction and Experiment Settings:* Lips reading seeks to decipher textual content via the visual cues provided by the speaker's lip movements. We opt the DVS-Lip dataset [1]and crop the event into 88x88. Other experiment settings are aligned with those of the main experiment. The performance is measured by the accuracy.\n\n\n| **Method** | Fixed | **Ours** |\n|:-------|:-------|:------:|\n| ResNet-18 | 16.23 | **18.50(+2.17%)**|\n| ResNet-34 | 18.14 |**19.08(+0.94%)**|\n\n### **(2) Human Gait Recognition**\n\n*Introduction and Experiment Settings:* Human gait recognition aims to determine human identities based on their walking patterns captured by the sensors. We select the EV-Gait dataset [2] and the input resolution is 128x128. \n\n\n| **Method** | Fixed | **Ours** |\n|:-------|:-------|:------:|\n| ResNet-18 | 88.40 | **89.15(+0.75%)**|\n| ResNet-34 | 84.90 |**89.10(+4.20%)**|\n\n\n\n### **(3) Camera Pose Relocalization**\n\n\n*Experiment Settings:* Camera pose relocalization aims to train several scene-specific neural networks to accurately relocalize the camera pose. We choose the IJRR dataset [3] and select the standard SP-LSTM [4] as our baseline. The input resolution is set as 180x240. The performance is measured by the distance error and the degree error. ME and AE denote the median and average error, respectively. We abbreviate meter and degree as m and e.\n\n\n||Shape Rotation||||Shape Translation||||\n|------------------------|--------------------|---------------------|---------------------|----------------------|------------------------|--------------------|---------------------|---------------------|\n| **Method**                 | **ME-m** | **ME-d** | **AE-m** | **AE-d** |**ME-m** | **ME-d** | **AE-m** | **AE-d** |\n| VGG-LSTM               | 0.065             | 26.014              | 0.081               | 38.398               | 0.177 | 8.623\t| 0.185\t| 13.176 |\n| VGG-LSTM **(+SpikeSlicer)**| **0.055**             | **21.292**              | **0.072**              | **22.666**               | **0.173**\t|**7.161**|\t**0.181**\t|**10.276**|\n\n\n### **(4) Object Detection in Real-world Environment**\n\n\n*Collection Settings:* To validate the effectiveness of our method in real-world scenarios, **we collected a small-scale, multi-object, event-based detection dataset under low-exposure conditions in an indoor setting.** We adopted a DAVIS-346 event-based camera with resolution 260x346 and recorded the dataset in the indoor under-exposure environment. Two volunteers participated in the recording and performed random body movements during the recording. The training set and testing set contains 125 and 18 event streams, respectively.\n\n*Experiment Settings:* We select three models as our baselines: YOLO-Tiny, YOLO-Nano and YOLO2. We crop the event as 320x320. We use the most commonly used mean average precision as our evaluation metric. We report the $\\mathbf{AP}$ of the person category.\n\n\n\n\n|   |  | Fix Slice|  |  | SpikeSlicer |  |\n|-----------|--------------|-------------|--------------|---------------|--------------|---------------| \n| **Method**    | $AP_{0.25}$  | $AP_{0.5}$  | $AP_{0.75}$  | $AP_{0.25}$ | $AP_{0.5}$  | $AP_{0.75}$ |\n| Yolo-tiny     | 48.46                | 46.80               | 38.24                | **54.58** **(+6.12)**    | **49.57** **(+2.77)**   | **44.87** **(+6.63)**   |\n| Yolo-nano     | 59.85                | 55.17               | 23.24                | **60.56** **(+0.71)**    | **55.33** **(+0.16)**   | **32.66** **(+9.42)**    |\n| Yolo-v2       | 62.21                | 59.65               | 46.45                | **62.62** **(+0.41)**    | **62.04** **(+2.39)**   | **48.38** **(+1.93)**    |\n\n\n\n### **(5) Summary**\nBased on the experimental results provided, we have further confirmed the efficacy of SpikeSlicer across various tasks. Additionally, SpikeSlicer has demonstrated positive outcomes in real-world scene detection. We will incorporate these results into future manuscript.\n\n\n\n\n\n---\n\n\n\n**Q2:** *The implementation of the code appears to be challenging, which may affect its reproducibility. The application of this algorithm in embedded systems seems to be constrained due to the use of Spiking Neural Networks (SNNs). Furthermore, it is unclear whether the code will be made publicly available.*\n\n**A2:** Thanks!\n\n*(1) Reproducibility:* We have provided detailed experimental settings in the appendix, including the network structure and algorithm details. We plan to open-source our code in the future and greatly appreciate your suggestion.\n\n*(2) Applicability of SNNs in Embedded Systems:* Indeed, numerous works have successfully deployed SNN algorithms on hardware devices, achieving high accuracy, remarkable operational efficiency, and low power consumption. For instance, the work by Yu et al. [5] published in Science Robotics, deployed SNN on the neuromorphic chip Tianjic [6], combined with an event-camera mounted on a quadruped robot to achieve efficient and low-energy locomotion in complex environments. Similarly, Roy et al. [7] deployed SNN on the Loihi chip to realize real-time event-based detection. There are also many other fields, e.g., EEG processing [8], robotics [9] , etc., where the deployment of SNN has been successfully implemented. Therefore, the application of SNNs in embedded systems is feasible.\n\n---\n\nDue to word limitations, the reference part is moved to the official comment.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uRx3RPdwii", "reviewer": "Reviewer_cS27", "summary": "This paper proposes to use spiking neural networks (SNNs) to slice the event stream in an adaptive manner before passing the voxelized events to the downstream inference model. The first step of the proposed method divides the input event stream into voxelized event cells with the same temporal interval. An SNN, constructed to have a scalar output, then takes the event cells recurrently. The timestamps when the SNN generates spikes are considered to be the slicing positions. The slicing SNN and the downstream inference model are trained together using the membrane potential-driven loss and the linear-assuming loss. The feedback-update strategy allows the two networks to be trained end-to-end. Extensive experiments on a toy example, object tracking, and recognition demonstrate that the proposed method can be easily integrated into existing models, bringing a noticeable performance improvement.", "questions": "I like the key idea and hope the paper can be accepted. \n\nI encourage the authors to respond to the third bullet point in the \"Weaknesses\" section. Additional experiments addressing the second bullet point are **not** expected as part of the rebuttal.", "limitations": "Yes, the authors have adequately addressed the limitations and potential negative societal impact of their work.", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper focuses on an interesting yet underexplored problem, which is to use a data-driven model to adaptively construct the event voxels. The proposed method is intuitive, and the key idea is convincing. It is clear that the authors have put in great effort in preparing this submission.\n\n2. The key technical contribution involves two parts. First, the feedback-update strategy allows supervision signals from the downstream ANN to back-propagate to the SNN. Additionally, the membrane potential-driven loss and the linear-assuming loss control the spiking time through the supervision of the membrane potential value. The two parts complement each other, leading to an end-to-end trainable model.\n\n3. Additionally, the paper also discusses how the hyperparameter $\\alpha$ can be tuned together with the SNN weights and analyzes the implication of different $\\alpha$ values to the spiking behavior. \n\n4. The experiments are very extensive. The SpikeSlicer has been validated on several event-based applications, demonstrating its prediction quality, efficiency, and the fact that it can be easily incorporated into existing models.", "weaknesses": "1. Despite the strengths above, the key design appears to be a bit simple. As a potential NeurIPS paper, this work is relatively weak on the technical sophistication and theoretical insights. However, this is complemented by extensive experimental evaluation and empirical analysis. \n\n2. While SNNs are efficient and consume less energy than ANNs, SNNs are also less capable than ANNs. Since the speed of the entire SNN+ANN prediction pipeline is going to be slow anyway, it may be worthwhile to investigate whether using an ANN as an event slicer can lead to better prediction quality.\n\n3. While the proposed losses are justified by proposition 1 and empirical analysis, it is unclear if the proposed feedback-update strategy is the best way to identify the desired trigger time $n^*$. In particular, it is unclear if the argmin operator can return any meaningful signal during the initial training stages.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1:** *While SNNs are efficient and consume less energy than ANNs, SNNs are also less capable than ANNs. Since the speed of the entire SNN+ANN prediction pipeline is going to be slow anyway, it may be worthwhile to investigate whether using an ANN as an event slicer can lead to better prediction quality.*\n\n**A1:**\nThank you for your suggestion! Firstly, it is indeed true that the current capabilities of SNNs are not on par with those of ANNs, and simulation speeds on GPUs can be slower for SNNs. However, our motivation for utilizing SNNs as a slicer is twofold: (1). SNNs are low-energy consuming, and (2). They have the potential for extremely high operational efficiency and speed when deployed on neuromorphic chips, a fact that has been substantiated by several studies [1,2,3]. Therefore, this paper takes this motivation as a starting point and proposes an event slicer based on SNNs. We fully acknowledge and appreciate your point that an ANN-based event slicer could potentially achieve higher performance. Given that the forward propagation in ANNs does not involve the concept of binary signals, the slicing process would require further design and consideration, which indeed warrants future investigation. Thank you again for your valuable suggestion!\n\n\n\n\n---\n\n\n**Q2:** *While the proposed losses are justified by proposition 1 and empirical analysis, it is unclear if the proposed feedback-update strategy is the best way to identify the desired trigger time $n^*$. In particular, it is unclear if the argmin operator can return any meaningful signal during the initial training stages.*\n\n**A2:**\nThank you for your question! To validate whether the proposed feedback-update strategy can serve a guiding role during the initial stages of training, we have visualized the training process of the SNN and presented it in **Figure 1** of the supplementary PDF rebuttal file. As anticipated, the training of the SNN exhibited fluctuations during the initial training stage, which might be attributed to the instability of the event quality obtained from dynamic slicing at this early phase. However, as training progressed, the loss of the SNN gradually stabilized and decreased, converging towards a desired outcome. Correspondingly, the slicing times progressively converged towards the desired spiking index. Therefore, although initial exploration may require several steps, our proposed training method is capable of offering effective guidance. We will also attempt further optimizations to enhance its efficiency. Thank you for your comment！\n\n\n\n---\n\n\n***Reference:***\n\n[1] Roy A, Nagaraj M, Liyanagedera C M, et al. Live demonstration: Real-time event-based speed detection using spiking neural networks. CVPR, 2023.\n\n[2] Yu F, Wu Y, Ma S, et al. Brain-inspired multimodal hybrid neural network for robot place recognition. Science Robotics, 2023.\n\n[3] Viale A, Marchisio A, Martina M, et al. Lanesnns: Spiking neural networks for lane detection on the loihi neuromorphic processor. IROS, 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We sincerely thank the reviewers for their thoughtful comments and feedback. We appreciate that all reviewers agreed that the idea of using spiking neural network (SNN) for dynamic event slicing is interesting and evaluated the paper with positive scores. Below, we address the primary concerns raised by the reviewers and summarize the revision:


- Reviewer ij7X suggested to provide comparison with the latest method. In response, we supplemented a recent tracking model and demonstrated the effectiveness of our method.
- HdTR suggested implementing a wider variety of tasks for testing. In response, we tried our best to incorporate more task types and supplemented **four** tasks. In particular, we conducted Lips Reading, Human Gait Recognition, Camera Pose Relocalization and Object Detection in **Real-world** Environment. 
- Reviewer C27 raised a good question, inquiring whether the proposed feedback-update strategy could serve a guiding role during the initial stages of training. In response, we visualized the training process of the SpikeSlicer, as depicted in Figure 1 of the submitted PDF file. By analyzing the changes in the training curves, we demonstrated the effectiveness of our proposed strategy.
- For real-world detection, we **collected a multi-object, event-based detection dataset** under low-exposure conditions. 
- We **submitted a rebuttal PDF** file which includes all the experiment results.
- We visualized examples from the newly added downstream task datasets, which are presented in Figure 2.
- We showcase the experimental setup for collecting real-world event dataset and visualized some samples, as shown in Figure 3.
- The experimental results for the supplementary tracking and downstream tasks (lip reading, gait recognition, camera pose relocalization, and detection) are compiled in Figure 2-3 and Table 1-4 for easy reference.


Thanks for your time and effort in reviewing our paper, your suggestions have greatly helped to enhance the article!

Meta-review: The authors present an approach for splitting event streams adaptively. The authors argue that typically, when dealing with input event streams, and when these events are processed by an algorithm that operates using frame-based input, an approach for slicing the input events is needed. This is usually done by splitting at uniform time intervals, or by splitting when the total spike density exceeds some threshold. The authors propose a system where a spiking neural network is actually responsible for adaptively deciding when to do the splitting, based on a feedback mechanism utilizing feedback provided by the ANN that does the top-level detection/classification task.

A concern I have with this work is that it assumes there is always a need to convert event streams into groups of spikes, or frames. This is only needed when you want to provide a frame-based representation as input to a more traditional neural network that operates based on the concept of a frame. When dealing with neuromorphic hardware and algorithms, you need to tone-down the concept of a frame that is associated with more traditional neural networks, in order to achieve efficiency.

Nevertheless, all reviewers seem to believe that the algorithm is useful when the event data is meant to be consumed by more traditional ANN algorithms that consume frame-like inputs. The reviewers indicate that the paper does a thorough experimental comparison and seems to demonstrate improved accuracy. 

The authors indicated in the rebuttal that they plan to release source code in the "future". This was in response to a concern expressed by one of the reviewers regarding reproducibility. After reading the paper I tend to agree with this reviewer comment. To ensure reproducibility of this work I would like to ask the authors that they release their source code, so that it is feasible for someone to reproduce their results. I believe this would also increase the impact of their work.

In summary there seems to be consensus among the reviewers, and therefore I recommend acceptance. As I indicated above, and as a condition of acceptance, I would like the authors to also release source code with the paper's publication (as they indicated in their rebuttal that they would do in the future) in order to make this work easily reproducible.
Decision: Accept (poster)