EGSST: Event-based Graph Spatiotemporal Sensitive Transformer for Object Detection
Authors: Sheng Wu
Hang Sheng
Hui Feng
Bo Hu
Venue: NeurIPS 2024 poster (2024)
Keywords: Event camera
Graph
Transformer
Object detection
Primary area: machine_vision
TL;DR: The EGSST framework, which is proposed for the rapid and efficient processing of event camera data for object detection, integrates a Graph Transformer structure with adaptive temporal attention, inspired by the dynamic response of human eyes.

Abstract: Event cameras provide exceptionally high temporal resolution in dynamic vision systems due to their unique event-driven mechanism. However, the sparse and asynchronous nature of event data makes frame-based visual processing methods inappropriate. This study proposes a novel framework, Event-based Graph Spatiotemporal Sensitive Transformer (EGSST), for the exploitation of spatial and temporal properties of event data. Firstly, a well-designed graph structure is employed to model event data, which not only preserves the original temporal data but also captures spatial details. Furthermore, inspired by the phenomenon that human eyes pay more attention to objects that produce significant dynamic changes, we design a Spatiotemporal Sensitivity Module (SSM) and an adaptive Temporal Activation Controller (TAC). Through these two modules, our framework can mimic the response of the human eyes in dynamic environments by selectively activating the temporal attention mechanism based on the relative dynamics of event data, thereby effectively conserving computational resources. In addition, the integration of a lightweight, multi-scale Linear Vision Transformer (LViT) markedly enhances processing efficiency. Our research proposes a fully event-driven approach, effectively exploiting the temporal precision of event data and optimising the allocation of computational resources by intelligently distinguishing the dynamics within the event data. The framework provides a lightweight, fast, accurate, and fully event-based solution for object detection tasks in complex dynamic environments, demonstrating significant practicality and potential for application.

EGSST: Event-based Graph Spatiotemporal Sensitive Transformer for Object Detection
Sheng Wu'
Hang Sheng' Hui 122
Bo Hu
School of Information Science and Technology, Fudan University
2 State Key Laboratory of Integrated Chips and Systems, Fudan University
Abstract
Event cameras provide exceptionally high temporal resolution in dynamic vision systems due to their unique event-driven mechanism. However, the sparse and asynchronous nature of event data makes frame- based visual processing methods inappropriate. This study proposes novel framework, Avent-based Graph Spa- tiotemporal Sensitive Transformer (EGSST), for the exploitation of spatial and temporal properties of event data. Firstly, a well-designed graph structure is em- ployed model event data, which not only preserves the original temporal data but also captures spatial details. Furthermore, inspired by the phenomenon that human eyes pay more attention to objects that produce significant dynamic changes, we design a Spatiotemporal Sensitivity Module (SSM) and an adaptive Temporal Acti- vation Controller (TAC). Through these two modules, our framework can mimic the response of the human eyes in dynamic environments by selectively activating the temporal attention mechanism based on the relative dynamies of event data, thereby effectively conserving computational resources. In addition, the integration of lightweight, multi-scale Linear Vision Transformer (LViT) markedly enhances processing efficiency. Our research proposes fully event-driven approach, effec- tively exploiting the temporal precision of event data and optimising the allocation of computational resources by intelligently distinguishing the dynamics within the event data. The framework provides a lightweight, fast, accurate, and fully event-based solution for object detection tasks in complex dynamic environments, demonstrating significant practicality and potential for application. The source code can be found at: EESST
Introduction
There is an increasing demand for devices capable of accurately capturing targets in high-speed dynamic scenes, such as autonomous driving, where traditional CMOS or CCD visual sensors often encounter motion blur and overexposure 2 3 In response, event cameras, which employ a novel event-driven mechanism, have garnered significant attention. Each pixel in event cameras operates independently, activating only upon detecting brightness change. This mechanism results in series of high-speed, asynchronous event streams with exceptionally high temporal resolution, which enables the rapid and precise capture of data (4)5).
Our objective processing the output data from event cameras effectively utilize their high temporal precision in detection tasks without inducing significant delays. However, the data format of event cameras entirely different from that of frame-based cameras, rendering existing object detection methods based on RGB images inapplicable (6). Some methods adopt conversion strategies, utilizing techniques such as temporal slicing to transform sparse data into dense formats, with
"Corresponding author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Event Data Stream
Graph Construction graph subgraphs)
Global Graph F Feature Map
Oka Yo Pc)
Prediction Outputs
GCN mintt
Multi-scale Linear VIT
Detection Head
TAC
SSM
Renaviing Subyraphs with (ane andie
Dynamic indicator
Fosture Maps
Osipets K,
Relative Dynamis Faiture
Couedinate sy eysition
Cooedinates with
Fadeo Mechsnion
ELM
Features
Adaption Galing
GAT
Mar harikin
Poxilin
Spatistenporal
Freture
Spotiotenpocal OOT rrgitiiioo
An overview of the proposed EGSST framework. The EGSST is event- bese- lightweight, and efficient framework designed for rapid object detection in event data. A graph is constructed from the data and divided into K connected subgraphs. These subgraphs are fed into Graph Convolutional Network (GCN) [22] and a SSM. The GCN processes the subgraphs that are not removed to produce global Graph Feature Map, which preserves both spatial and temporal information. The SSM assesses the dynamics of the entire graph and outputs a dynamic feature indicator, which includes the dynamics of each subgraph and the aggregated dynamics obtained through a Graph Attention Network (GAT) 233 The TAC is activated based on the output from the SSM to enhance focus on the temporal dimension to feed the graph feature maps directly into Multi-scale Linear ViT [24]. Finally, a detection head, such as RT-DETR 25 or YOLOX 26 is employed to generate prediction outputs.
Figure 1:
the aim of leveraging architectures like Convolutional Neural Networks (CNNs) or Transformers 7 8 9 10 [I] Unfortunately, methods employing such conversion strategies often result in a loss
of temporal precision. Another direct processing strategies involve methods such Graph Neural
Networks (GNNs) [2 13 14. [5] or Spiking Neural Networks (SNNs) [6. I7
I8
19 20 21].
which analyze the event stream directly, thereby preserving spatiotemporal structure and accuracy. However, the direct processing strategies often suffer from computational efficiency. The latest hybrid strategies strive combine the strengths of previous methods, enhancing performance while maximizing retention of temporal precision. Although numerous existing strategies have made significant progress in the extraction of event data features, challenges remain in effectively managing data across spatial and temporal dimensions 12 [4].
of computational resources when processing data across both spatial and temporal dimensions. Because current models either apply a single algorithm to both dimensions simultaneously [12 20 or use different algorithms for each dimension and then combine the results (27). Importantly, processing in the temporal dimension typically consumes significantly more computational resources than processing in the spatial dimension. Consequently, the continuous processing of temporal data is likely to result in the inefficient utilization of computational resources, especially for slower-moving objects where spatial resolution may already be sufficient.
Existing methods may result in the inefficient
In order to address the aforementioned issue of inefficiency, we have drawn inspiration from the dynamic perception of human eyes. The human visual system naturally prioritizes rapidly moving objects within the visual field while de-prioritizing slower ones-an adaptive feature that enhances responsiveness and efficiency in dynamic environments. Ideally, artificial models can emulate this trait by preferentially processing faster-moving objects in the temporal dimension, especially when managing large volumes of continuous spatiotemporal data. Therefore, we try to develop a novel algorithm that is closer to the selective attention mechanism of the human eye, which could potentially enhance both the efficiency and efffectivnees of event- vision systems.
We propose a novel spatiotemporal fusion graph transformer framework, designed to fully leverage the powerful capability of graph to process unstructured data while also reducing unnecessary computations in handling spatio-temporal data. Firstly, the framework employs a graph structure to

---

model event data, thereby maintaining the original temporal fidelity and capturing crucial spatial details. A key aspect of our framework the SSM module, which leverages the graph's 's ability to process unstructured data and aggregates features to effectively discern the relative dynamics of objects. This differentiation provides critical basis for determining whether to continue processing in the temporal dimension. Subsequently, the adaptive TAC is introduced, which dynamically adjusts its activation based on the insights from the SSM. The TAC is designed to enhance the processing of highly dynamic events while reducing resource use in scenarios with low dynamics, Furthermore, the integration of a lightweight multi-scale LVTT markedly enhances the processing efficiency of our system. The principal contributions of our research are as follows:
Our model introduces a novel and efficient graph processing method by pioneering the use of connected subgraphs in the context of event cameras. This technique not only preserves the temporal data completely but also enhances the effective and precise focus on targets within the event data.
We introduce SSM and TAC to mimic the human eye's perception of dynamics and integrate them into the Graph Transformer framework for efficient object detection in event data,
Our model integrates Graph and Transformer technologies to enhance object detection tasks in a fully event-based manner. It is designed to be lightweight, fast, and precise, representing a novel approach that leverages the strengths of both technologies for improved performance.
2 Related Work
The unique event-driven mechanism of event cameras offers considerable potential for high-speed motion detection in dynamic environments. However, the inherent sparsity and unstructured nature of the data generated by these cameras pose significant challenges to conventional image processing techniques 28). In response, researchers have transitioned from a singular processing strategy to a hybrid approach, integrating traditional and innovative methods.
Conversion Processing Strategy involves transforming event data into dense frame- basee data, making it compatible with conventional visual processing algorithms. Systems like the E2VID [29] utilize CNNs to convert asynchronous event data into video frames through time slicing. Other approaches generate continuous optical flow and intensity estimation images from event streams [30, or incorporate attention mechanisms with recurrent and convolutional networks to enhance spatio-temporal feature extraction [1 These strategies extend the applicability of event camera data but often at the cost of reducing its high temporal resolution [3 B3 34].
Direct Processing Strategy focuses on preserving the original asynchronous characteristics of event data to maintain its sparsity and high temporal precision. This includes direct modeling of the event stream for real-time object tracking (35) and employing spiking neural networks (SNNs) for gesture recognition 77. Additionally, models like AEGNN [12] leverage GNNs and efficient updating strategies to retain asynchronous temporal features. While these methods effectively utilize the unique properties of event data, they face challenges in handling large volumes of sparse data, highlighting the need for further innovations in architecture (36 20
Hybrid Processing Strategy explores a combination of conversion and direct processing methods, or the integration of multiple model frameworks [37 38
g 27 For example, merging GNNs with CNNs, and integrating Transformers with recurrent neural networks, effectively captures spatio-temporal features from asynchronous events while maintaining high resolution and enhancing increase data processing efficiency by jointly extracting features. Building on these studies, recent work, RVT (27). integrates Transformers with recurrent neural networks, significantly improving processing efficiency and accuracy. However, current models, despite their commendable performance and efficiency, cannot effectively differentiate dynamic data, leading to unnecessary computations in the time dimension.
39
40
adaptability [38] 39]. Methods like MatriXLSTM [40] and RED g
Our research lies within the hybrid processing strategy methods. The proposed framework leverages graph capabilities to manage irregular data and incorporates temporal attention mechanism respon- sive to data dynamics. It also integrates a lightweight linear visual Transformer to extract temporal and spatial information from event data. This event-based framework designed to provide a fast, efficient, and accurate object detection solution, specifically tailored to event cameras.
3

---

3 Methodology
The proposed framework utilizes the Graph Transformer to extract graph- based features from un- structured event data and integrates both spatial and temporal attention mechanisms. As illustrated in Figure | the architecture and key components are depicted, providing a visual overview of how these elements are integrated. This section will outline the critical steps involved in the framework.
3.1 Graph Construction
Each event captured by an event camera records both spatial information and precise temporal details, crucial for dynamic scene analysis. That is, each event is recorded as
evend, = (I,, Vr, Pt), i E
where 24 U4 locate the pixel and i, represents the timestamp with microsecond precision. The coordinates z; and y/ as well the timestamps & are completely asynchronous and occur randomly, reflecting the event-driven nature of the data capture. The polarity p; E {-1,1) denotes whether there is a decrease (for 1) or increase (for 1) in pixel brightness. N 21, N is the set of indexes of events.
A braph-based representation is employed to capture irregular spatiotemporal relationships between events. Firstly, in order to enhance the stability of the model and prevent data bias, especially for timestamps with otherwise large values, it is necessary to normalize the timestamp for each event by the equation ! 3- (1, ta), i E N, where to = minteN (t:} and a normalization factor. Each event event; then generates vertex v; (æz, Vs. & Pt), and V [2,,"2.... .2...... of vertices. The position coordinates of each vertex can be represented as c, = [Tt, Us: fT
is the set
To establish the graph edges, consider every pair Cy). An edge eij is added to the set E of edges the conditions are met,
1, if ||c - c S R,
Gij - (0, otherwise,
indicating the connectivity based on the predefined spatiotemporal distance threshold R. The resulting graph, G EV, effectively encapsulates the essential spatialtemporal properties of events.
3.2 Connected Subgraphs Construction
Event cameras produce data triggered by pixel brightness changes. Dense events occur at object contours where brightness exhibits significant variation, whereas the main body sees fewer, sparse events due to subtle changes. Transforming this event data into graph G, areas with dense events display numerous edges, whereas sparser areas show fewer edges. This pattern of data density leads us to the concept of connected graphs in graph theory, which describes the interconnectivity between the nodes of an undirected graph. The contour regions, due to their high number of edges, form several large connected subgraphs with a substantial number of nodes. contrast, the main bodies, which have fewer edges, result in connected subgraphs with fewer nodes. By filtering the data based on the number of nodes, the majority of the retained connected subgraphs attain new physical meaning, indicating the contiguous edges of object contours. The entire graph G can be divided into K connected subgraphs, G = (Vv, Ex), which satisfy
D = (Gy: Go G, = K...
However, it should be noted that the connected subgraphs not in one-to-one correspondence with the objects Dj in the event data. Consequently, the number of selected subgraphs, K, is typically greater than the number of real objects, L.
D.D. 1= L
4
By filtering the number of nodes contained in the connected subgraphs, we can filter out subgraphs containing too few nodes, thus realizing downsampling. Based on preliminary experimental tests with the dataset used, processing 10,000 events can retain approximately 73% of the events, which

---

Original
Dynamic
High
(a)
Rolative Value
(b)
Low
Figure 2: Dynamic Visualization of the SSM. Each image is generated from 10,000 event points, causing slight blurring. However, connected subgraphs effectively filter out background noise, preserving only relevant objects. (a) The scene shows low relative dynamic, hence the distinction is pronounced. (b) The truck the right accelerates overtake, while the car the left moves slower, making the truck's relative values significantly higher.
helps reduce noise interference. In contrast to conventional uniform downsampling, this approach avoids the loss of valid features associated with nodes. Moreover, the scope of event data that requires focus can be selected in more effective and rational manner.
After constructing connected subgraphs, we utilize a multilayer GCN that integrates node feature information with graph topology, efficiently learning node representations. The GCN preserves and fuses spatial and temporal features globally, producing a global graph feature map.
3.3 Spatiotemporal Sensitivity Module (SSM)
The Spatiotemporal Sensitivity Module (SSM) is core component of our framework. It mimics the visual characteristics of the human eye to quantify object dynamics in event data. Since event data differs from traditional images, directly quantifying object dynamics using displacement speed is challenging. Thus, we propose Event Global Motion (EGM) and Event Local Motion (ELM), applied to the global graph G and subgraphs Gk, respectively.
We first define the following metrics to separately quantify the global and local dynamics:
For global motion, we define:
EGM [[N, AI",,
(5)
where N = JV| represents the number nodes in the global graph, and At" max mex. represents the maximum time difference among nodes.
-t711
For local motion, we define:
ELMP
f(r,, 8t%),
(6)
where ny represents the number nodes involved the measurement of the k-th subgraph, and 81g is the maximum time difference within the subgraph. While the function could be designed to take more complex forms to capture intricate relationships between these variables, we opt for a simple ratio-based method, f(z, y) z/y. This choice is motivated by the desire to balance simplicity and computational efficiency. The ratio directly reflects the relationship between the number of events and the temporal spread in the subgraph, providing an intuitive measure of local motion dynamics. By using this straightforward formulation, we ensure that the computation remains tractable, especially when dealing with large-scale event data, while still effectively capturing the key dynamics of interest.
For instance, in a traffic scenario, event data may include dynamic entities such as vehicles and pedestrians, as well static background elements like trees and fences. EGM quantifies the dynamics of the entire event data, representing the overall level of motion in the environment, while ELMs describes the motion level of individual subgraphs. Given the high temporal resolution of
5

---

event cameras, we can assume that any object responds to events almost instantaneously. As an object's speed increases, more pixels detect brightness changes, generating more events, which increases ne and decreases S. leading to a significant increase in F.
To analyze the relationship between the dynamics of each subgraph relative the overall environment, we introduce the following metric:
ELMM
H
h4
k 1==1,2 1..,,.... K
EGM
which quantifies the relative motion of each subgraph with respect to the global dynamic level. During subgraph construction, distance thresholds and limitations on adjacent nodes are set divide large objects into multiple subgraphs, thus preventing a single subgraph from dominating the motion representation of a large object.
While all nodes in each subgraph should reflect relative dynamics, assigning the same features to every node unnecessary as would result in redundant computations. We aggregate the spatiotemporal coordinates of each subgraph into representative coordinates as follows:
(8) where V denotes the set of spatiotemporal position coordinates for subgraph nodes, and π is function for aggregation 12 AT, such mean, max, or min. These functions are chosen because they are insensitive to the varying number of nodes in each subgraph, ensuring that the aggregation process remains consistent regardless of subgraph size. We then use the K-nearest neighbors algorithm to connect these new representative coordinates, forming new set of edges E, mapping features h2 to the corresponding representative coordinates, and building new set of vertices V. This results in a new graph 9 (V,E), E), which significantly reduces the number of nodes and decreases computational complexity and processing delay.
=VV[): A = 1, K}},
In the newly constructed graph, each node represents an aggregated subgraph with a relatively sparse distribution in the spatiotemporal domain, necessitating the enhancement suppression of relationships between nodes. We introduce inter-subgraph attention using a Graph Attention Network (GAT) 23 capture relationships among nodes. The GAT enables nodes to adaptively aggregate information based on the importance of their features and their neighbors, strengthening the connections between nodes representing the same object while weakening those representing different objects. The GAT's attention mechanism also ensures that smaller objects are not overlooked and evaluates their significance in the overall environment. The aggregated values are used as the final output of the SSM for assessing relative motion. Figure sshows some example results, illustrating how the spatiotemporal sensitivity module captures the relative dynamics in different regions.
3.4 Temporal Activation Controller
Upon evaluation of the output from the SSM, a determination is made as to whether to enhance the temporal foeus by activating the TAC. The TAC processing comprises two parts: firstly, a simple attention-like method is employed to dynamically weight and selectively emphasise the temporal dimension; secondly, the Query, Key, and Value (QKV) are reconstructed.
In the first part of the TAC, we define a set F = }FF}, where F; represents the stream of graph feature maps over time t, and F is the collection of these streams. We aggregate all features within the set using the formula Fs, Aggregation comprehensive representation of the spatiotemporal data. Subsequently,
to generate the Global Spatiotemporal Feature,
F.t, which serves employ an Adaptive Gating Mechanism to adjust these features at each timestamp, generating gating signals from F.t for each F,. This mechanism either enhances or suppresses the features based on the global feature, thereby maintaining temporal integrity while adapting to the sequence context. Finally, adaptive fusion technique modulates the integration between the original and enhanced features, producing outputs suitable for subsequent processing.
In the second part of TAC, different sources generate the QKV matrices,
Q (F,(ss), K, V W (F).
The global spatiotemporal feature F.t is employed to generate the Query (Q), which offers a global perspective when assessing the relevance of each timestamp to the entire sequence. Mcanwhile, the

---

Multi-scale Linear VII:
Enchanced
Stage
Stage
Stage 3
Stage 4
1
Ag
Lg
Input
ResCam
Tif MBCom
MBCcnvv
TiA
Delection Iead
(1 incar)
R 32, R1
[B JUW]
BB 192, W2]
[B. 120 JML WM]
[a 256, 9801
Activate TAC
Data Elaw
Remowable
Enchanced CNN:
Activated TAC:
Mapss
A Alention Network
G Cating Network
Cam/3k13
Uysaopple
Dwr
Cuas Ixl
Cunvisd
lape
S11
Nara
Ságeoiid
Siguoiil
ShipCon omv
Figure 3: The flowchart of the Multi-scale Linear VIT. This diagram shows the stages of the Multi- scale Linear ViT, with the removable Enhanced CNN and Activated TAC modules. The Enhanced CNN processes input data through convolutional and normalization layers before passing it to the ViT stages. Activated TACs Stage and Stage 4 optimize temporal processing and balance efficiency. The data is then sent to the detection head for final object detection.
utilisation of the aggregated outputs each timestamp in the first part as the source of Keys (K) and Values (V) generation helps to reflect the state of each timestamp with greater accuracy. This method of employing diverse generation sources merges global and local insights, thereby bolstering the model's is capability to process complex spatiotemporal data within self-attention architecture.
It is worth mentioning that TAC is not module but component, it theoretically be loaded on any model focusing on the extraction of spatial features, greatly improving scalability.
3.5 Multi-scale Linear VIT
The original Vision Transformer (ViT) 42 does not allow for multi-scale processing, resulting in significant performance degradation compared to models equipped with such capabilities [43] 44 45
To address this issue, our enhanced VIT module supports multi-scale processing and incorporates the TAC, which improves the model's ability to handle spatial and temporal dimensions. To reduce the computational load and increase processing speed, we adopt linear transformations simplify the self-attention mechanism. Drawing upon techniques from models like Efficient VIT (24), which reduce the computational complexity of self-attention from ( to approximately O(N), our ViT module adapts these cost-reduction strategies to fit within our framework.
Experiments
In this section, we introduce the two datasets utilized, the evaluation metrics, and the implementation details of models. We train the baseline model, EGSST-B, and the extended model, EGSST-E, and compare their performance with other state-of-the-arr models applied to both datasets. Detailed ablation studies then performed assess the impact of various components of models. Finally, to verify the scalability of the models, we train them using varying numbers of events, obtain their corresponding weights, and analyze these weights in differently configured models.
4.1 Datasets and Evaluation Metrics
Two complex event camera datasets from traffic scenarios are employed in the experiments: the Genl Automotive Detection Dataset [46] and the Megapixel Automotive Detection Dataset 9
The Genl Dataset comprises over 39 hours of event video from urban, highway, and rural settings, with resolution of 304 240 pixels. The dataset includes manual annotations of pedestrians and
7

---

cars, with over 255,000 labels at a frequency of I to 4 Hz, making it ideal for testing object detection, tracking, and optical flow algorithms in automotive environments.
The 1Mpx Dataset comprises over 14 hours of high-resolution (1 megapixel) event video, annotated with 25 million labels for cars, pedestrians, and two-wheelers, suitable for developing advanced detection models in dynamic traffic conditions.
Three primary evaluation metries employed in the experiments, namely the total number of parameters, the mean Average Precision (mAP@0..:0.955) using the COCO toolbox [47], and the mean time per batch (batch size 1) for detection. These metrics assess the models' complexity, precision, and efficiency, respectively, in real-time applications.
4.2 Implementation Details
The framework proposed in this study is developed using Python 3.9 and PyTorch 2.0, with graph processing powered by the advanced PyTorch Geometric library 48). To enhance the scalability and parallelism of subgraph processing, modifications are made to the underlying libraries to achieve complete parallelization of graph processing, thus fully leveraging GPU computational capabilities. The models are trained on RTX3090 GPUs using the Lightning framework. We employ the Adam optimizer 49 coupled with the OneCycle learning rate schedule [50], which includes 100 warm-up up iterations followed by cosine decay starting from the maximum learning rate. The training batch size set at 8, with an initial learning rate of 1e-6.
4.3 Dynamic Label Augmentation
In our model, we use batch segmentation method based on fixed number of events, which requires generating corresponding labels for inputs with varying numbers of nodes. However, the data generation speed of event cameras is extremely high, while the number of labels available in existing datasets is relatively limited. For instance, current datasets such as Gen cannot effectively provide sufficient labels to match the rapidly collected is necessary to develop effective approach to increase the number of available labels.
event data. To address this mismatch,
extend the labeling by adding a fixed time window before and after each label, mapping all the data within that time frame to the same label 112 27 [T 52) While this method can indeed increase the number of labels under typical conditions, it can introduce significant labeling errors when applied to fixed-number event- based batch segmentation, especially in dynamic environments with substantial variability.
common approach
To address this issue, we propose a dynamic label augmentation method. This approach dynamically adjusts the labeling precision based the time span over which events are captured, aiming to expanding the number of labels while enhancing labeling accuracy. Specifically, in the dynamic label augmentation method, longer time span for collecting a fixed number of events indicates slower target motion, allowing for an expanded time window to capture more labels for the current target. Conversely, a shorter time span suggests faster target motion, necessitating a smaller time window to maintain accurate labeling. More details can be found in the Appendix C
4.4 Benchmark Comparisons
Our baseline model, EGSST-B, achieved an impressive processing time of only 2.4 milliseconds on the RTX 3090. However, this result was obtained without considering GPU power consumption. To ensure fair comparison with other models, we conducte additional tests on the T4 GPU, which has performance comparable to the Titan Xp and RTX 1080Ti. The ASTMNet and RED models are tested the Titan Xp. while the AEC model was evaluated on the RTX 1080Ti.
Table compares the performance of various target detection methods on the Genl and Mpx datasets. Among these benchmarks, our models EGSST-B and EGSST-E outperform the rest. EGSST- achieves a 49. 49.6% mAP on the 1 Mpx dataset, demonstrating exceptional capabilities; EGSST-B processes in just 4.6 milliseconds on the Genl dataset, significantly outperforming other models in terms of efficiency and real-time processing. With parameter counts of 3.5M and 12.3M, respectively, these models achieve high performance while being more streamlined compared to traditional approaches. This optimization in parameter efficiency particularly crucial for deployments on resource-limits platforms.
8

---

Table 1: Comparison of Different Event-Based Vision Methods. The results of our methods are obtained from experiments involving 10,000 events. Our methods ending in -Y utilize the YOLOX 26 detection head instead the RT-DETR [25] method. A star indiccatss that this information not directly available and can be estimated based on modules in published articles.
Genl
Mpx
Method
Backbone
mAP (%6 f
Time
MAP
Time
Params
(ms) &
(%) f
(ms) 4
(M)
RRC-Events
32
CNN
30.7
21.5
34.3
46.4
>100
AEGNN 12
GNN
16.3
20.0
Spiking DenseNet 20
SNN
18.9
8,2
ERGO-12 53
Transformer
50.4
69.9
40.6
100.0
59.6
RED
CNN RNN
40.0
16.7
43.0
39.3
ASTMet (37
CNN + RNN
46.7
35.6
48.3
72.3
> 1000
AFC + DETR 54
44.5
7.7
45.9
20.7
>40*
AFC YOL.0v5
54
47.0
3.9
48.4
13.8
>40*
RVT-B 27
Transformer RNN
47.2
10.2
47.4
11.9
18.5
GET-T
Transformer RNN
47.9
16.8
48.4
18.2
21.9
S4D-ViT-B
Transformer SSM
46.2
9.4
46.8
10.9
16.5
S5-ViT-B [5]]
Transformer SSM
47.4
8.16
47.2
9.57
18.2
EGSST-B (ours)
GNN Lin
44.6
4.6
45.4
5.1
3,5
EGSST-E (ours)
GNN ViT
49.6
6.0
50.2
6.3
12.3
EGSST-B-Y (ours)
GNN LinearViT
43.9
3.7
44.1
5.0
3.1
EGSST-E-Y (ours)
GNN LinearViT
47.8
4.2
48.3
5.3
10.4
The EGSST model leverages a fully event-based architecture that makes full use of the rich spa- tiotemporal data generated by event cameras. The model achieves low processing latency through effective data downsampling, efficient batch processing, and the use of techniques that mimic the dynamic characteristics of the human eye. Additionally, the application of linear feature extractors further enhances its processing efficiency. These aspects enable the model to effectively utilize global information while minimizing computational demands, making it well-suited for object detection tasks using event cameras.
Ablation Studies
All experiments are conducted under identical environmental and equipment conditions. Given the similar performance observed on the MMpx and Genl datasets, the experiments described in this section are based on the Genl dataset unless otherwise stated. Moreover, since both of our models demonstrated similar test results, only the EGSST-E model is used for further analysis.
Table 2: Impact of applying TAC. The 'TAC Adaptive' refers to the integration of SSM with TAC, allowing for adaptive adjustments based on the data state.
Condition
mAP (%)
Time (ms)
Params (M)
TAC Inactive
42.1
4.3
10.2
TAC Active
51.5
11.1
16.3
TAC Adaptive
49.6
6.0
12.3
The SSM and TAC, integrated in series within our framework, are the reason why we conduct comparative experiments with these two components together. The presence of SSM dynamically activates TAC, thereby causing the parameters involved in the forward computation to vary dynami- cally as well. For quantitative analysis, we calculated the average amount of parameters under the condition of 10,000 event inputs across our test dataset. As shown in Table 2 the application state of TAC significantly impacts evaluation metrics. By activating TAC at appropriate times, SSM avoids unnecessary computations and time loss, which not only enhances model efficiency but also confirms the effectiveness SSM. Although the full application of TAC can maximize mAP accuracy, the additional time and computational costs involved not economical.

---

Table 3: Comparison after incorporating CNN. The increase with the addition of the CNN is shown in parentheses.
Method (with CNN)
mAP (%)
Time (ms)
Params (M)
EGSST-B
44.6 (+0.4)
(+0.1)
3.51 (+0.02)
EGSST-E
49.6 (+0.5)
6.0 (+0.1)
12.34 (+0.02)
Removable Convolutional Neural Network. Our framework is event-based and does not CNN for feature extraction. Nevertheless, it has been observed that integrating simple CNN module enhances detection performance. To operationalize this module, certain number event data must be standardized across the temporal dimension to form an image-like representation. Table shows that this CNN integration increases mAP with minimal impact on processing time and parameters. Consequently, we have incorporated removable CNN layer into the model, allowing for flexibility revert a fully event-driven configuration if needed.
Gen1 Dataset
1Mpx Dataset
Figure 4: Prediction Results. Due to the low accumulated event count, the visualizations appear somewhat blurred. Nevertheless, our model effectively identifies objects within these sparse events, demonstrating its robustness and efficacy.
Discussion and Conclusion
We present lightweight, efficient, and accurate event-based object detection framework tailored for event cameras. Our model shows strong detection performance and scalability, highlighting its practi- cal application potential. Future work will extend this approach object tracking and other advanced visual tasks, as well as explore deployment optimizations, such as enhancing execution speeds and converting models to ONNX or TensorRT formats. Deploying ViTs relatively straightforward, while GNNs pose greater challenges due to complex graph processing requirements. Although progress has been made, further work is necessary to improve the deployment efficiency of GNNs.
Furthermore, we plan to investigate the integration of event data with RGB frames in a multimodal model 55 56 [7 which could enhance the robustness and accuracy of visual recognition tasks under varied and dynamic conditions. This multimodal approach promises to leverage the strengths of both sensor types deliver superior performance in complex environments.
Acknowledgments and Disclosure of Funding
This research was supported by the National Key Research and Development Program under grant number 2024YFE0200703.
10

---

References
[1] Mathias Gehrig, Willem Aarents, Daniel Gehrig, and Davide Scaramuzza. Dsec: stereo event camera dataset for driving scenarios. IEEE Robotics and Automation Letters, 6(3):4947-4954, 2021.
[2] Amos Sironi, Manuele Brambilla, Nicolas Bourdis, Xavier Lagorce, and Ryad Benosman. Hats: Histograms of averaged time surfaces for robust event-based object classification. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1731- 1740, 2018.
[3] Alex Zihao Zhu, Dinesh Thakur, Tolga Özaslan, Bernd Pfrommer, Vijay Kumar, and Kostas Danilidis.. The multivehicle stereo event camera dataset: An event camera dataset for 3d perception. IEEE Robotics and Automation Letters, 3(3):2032-2039, 2018.
[4] Guillermo Gallego, Tobi Delbrücck, Garrick Orchard, Chiara Bartolozzi, Brian Taba, Andrea Censi, Stefan Leutenegger, Andrew J. Davison, Jörg Conradt, Kostas Daniilidis, and Davide Scaramuzza. Event-based vision: A survey. IEEE Transactions on Pattern Analysis and doi: U0.100//PAMM.2000000000
Machine Intelligence, 44(1):154- 2022.
[5] Bongki Son, Yunjae Suh, Sungho Kim, Heejae Jung, Jun-Seok Kim, Changwoo Shin, Keunju Park, Kyoobin Lee, Jinman Park, Jooyeon Woo, et al. 4.1 a 640 480 dynamic vision sensor with a 9um pixel and 300meps address-event representation. In 2017 IEEE International Solid-State Circuits Conference (ISSCC), pages 66-61 67. IEEE, 2017.
[6] Daniel Gehrig, Antonio Loquercio. Konstantinos G Derpanis, and Davide Scaramuzza. End- to-end learning of representations for asynchronous event-based data. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5633-5643, 2019.
[7] Arnon Amir, Brian Taba, David Berg, Timothy Melano, Jeffrey McKinstry, Carmelo Di Nolfo, Tapan Nayak, Alexander Andreopoulos, Guillaume Garreau, Marcela Mendoza, et al. A low power, fully event-based gesture recognition system. Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7243-7252, 2017.
[8] Javier Caari,,,,, Daniel Gehrig, and Davide Scaramuzza. Learning monocular dense depth from events. In 2020 International Conference on 3D Vision (3DV), pages 534-542 IEEE, 2020.
[9] Etienne Perot, Pierre De Tournemire, Davide Nitti, Jonathan Masci, and Amos Sironi. Learning detect objects with a megapixel event camera. Advances Neural Information Processing 2020.
Systems, 33:166339-1663
[10] Stepan Tulyakov, Daniel Gehrig, Stamatios Georgoulis, Julius Erbach, Mathias Gehrig, Yuanyou Li. and Davide Scaramuzza. Time lens: Event-based video frame interpolation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16155-16164, 2021.
[11] Vaswani. Attention is all you need. Advances Neural Information Processing Systems, 2017.
[12] Simon Schaefer, Daniel Gehrig, and Davide Scaramuzza. Aegnn: Asynchronous event-based graph neural networks. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 12371-12381, 2022.
[13] Zhenggi Li, Simon Niklaus, Noah Snavely, and Oliver Wang. Neural scene flow fields for space-time view synthesis of dynamic scenes. Proceedings of the IEEE/CVF Conference on Computer Vision and Patlern Recognition, pages 6498-6500 2021.
[14] Thomas Dalgaty, Thomas Mesquida, Damien Joubert, Amos Sironi, Pascal Vivet, and Christoph Posch. Hugnet: Hemi-spherical update graph neural network applied to low-latency event-based optical flow. Proceedings of the IEEF/CVE Conference on Computer Vision and Pattern Recognition, pages 3952-3961, 2023.

---

[15] Dominik Eisl, Fabian Herzog, Jean-Luc Dugelay, Ludovic Apvrille, and Gerhard Rigoll. In- troducing framework for single-human tracking using event based cameras. In 2023 IEEE International Conference on Image Processing (ICIP), pages
IEEE, 2023.
[16] Samanwoy DaasidDastidar and Hojjat Adeli. Spiking neural networks. International journal of neural systems, 19(04):295-300 2009.
[17] Jun Haeng Lee, Tobi Delbruck, and Michael Pfeiffer. Training deep spiking neural networks using backpropagation. Frontiers in neuroscience, 10:228000, 2016.
[18] Sumit B Shrestha and Garrick Orchard. Slayer: Spike layer error reassignment in time. Advances in neural information processing systems, 31, 2018.
[19] Mathias Gehrig, Sumit Bam Shrestha, Daniel Mouritzen, and Davide Scaramuzza. Swent-based angular velocity regression with spiking networks. In 2020 JEEE International Conference on Robotics and Automation (ICRA), pages 4102-4202. IEEE, 2020.
[20] Loic Cordone, Benoît Miramond, and Philippe Thierion. Object detection with spiking neural networks on automotive event data. In 2022 International Joint Conference on Neural Networks (IJCNN), pages 1-8. IEEE, 2022.
[21] Jiqing Zhang, Bo Dong, Haiwei Zhang, Jianchuan Ding, Felix Heide, Baocai Yin, and Xin Yang. Spiking transformers for event- bastd single object tracking. In Proceedings of the IEEE/CVF conference Computer Vision and Pattern Recognition, pages 8801-8810, 2022.
[22] Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint 16090009770 2016.
[23] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. arXiv preprint arXiv: 1710. 10903, 2017.
[24] Han Cai, Junyan Li, Muyan Hu, Chuang Gan, and Song Han. Efficientvit: Lightweight multi-scale attention for high-resolution dense prediction. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 17302-17313, October 2023.
[25] Yian Zhao, Wenyu Lv, Shangliang Xu, Jinman Wei, Guanzhong Wang, Qingging Dang, Yi Liu, and Jie Chen. Detrs beat yolos on Conference on Computer Vision and Pattern Recognition, pages 16965-16974, 2024.
object detection. In Proceedings of the IEEE/CVF
[26] Z Ge. Yolox: Exceeding yolo series in 2021. arXiv preprint arXiv:2107.08430 2021.
[27] Mathias Gehrig and Davide Scaramuzza. Recurrent vision transformers for object detection with event cameras. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 33884-13893, 2023.
[28] Xu Zheng, Yexin Liu, Yunfan Lu, Tongyan Hua, Tianbo Pan, Weiming Zhang, Dacheng Tao, and Lin Wang. Deep learning for event- based vision: A comprehensive survey and benchmarks. arXiv preprint arXiv:2330..08:: 088990 2023.
[29] Cedric Scheerlinck, Henri Rebecq, Timo Stoffregen, Nick Barnes, Robert Mahony, and Davide Scaramuzza. Ced: Color event camera dataset. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition Workshops, pages 0-0, 2019.
[30] Patrick Bardow, Andrew Davison, and Stefan Leutenegger. Simultancous optical flow and intensity estimation from event camera. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 884-892, 2016.
[31] Marco Cannici, Marco Ciccone, Andrea Romanoni, and Matteo Matteucci. A differentiable recurrent surface for asynchronous event-based data. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XX 16, pages
136-152. Springer, 2020.
[32] Nicholas FY Chen. Pseudo-labels for supervised learning on dynamic vision sensor data, applied to object detection under ego-motion. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops. pages 644-653, 2018.
12

---

[33] Massimiliano lacono, Stefan Weber, Arren Glover, and Chiara Bartolozzi. Towards event-driven object detection with off- shelhee-shelf deep learning. In 2018 JEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1-9. IEFE, 2018.
[34] Zhuangyi Jiang, Pengfei Xia, Kai Huang, Walter Stechele, Guang Chen, Zhenshan Bing, and Alois Knoll. Mixed frame-levent-drive fast pedestrian detection. In 2019 International Conference on Robotics and Automation (ICRA), pages 8332-83388 IEEE, 2019.
[35] Guillermo Gallego, Jon EA Lund, Elias Mueggler, Henri Rebecq, Tobi Delbruck, and Da- vide Scaramuzza. Event-based, 6-dof camera tracking from photometric depth maps. IEEE transactions on pattern analysis and machine intelligence,
2017.
[36] Yijin Li. Han Zhou, Bangbang Yang. Ye Zhang, Zhaopeng Cui, Hujun Bao, and Guofeng Zhang. Graph-base based asynchronous event processing for rapid object recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 934-943, 2021.
[37] Jianing Li, Jia Li, Lin Zhu, Xijie Xiang, Tiejun Huang, and Yonghong Tian. Asynchronous spatio-temporal memory network for continuous event-based object detection. IEEE Transac- tions on Image Processing. 31:2975-2987, 2022.
[38] Nico Messikommer, Daniel Gehrig, Antonio Loquercio, and Davide Scaramuzza. Event-based asynchronous sparse convolutional networks. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part VIII 16, pages 415-431. Springer, 2020.
[39] Ze Huang, Li Sun, Cheng Zhao, Song Li, and Songzhi Su. Eventpoint: Self-supervised interest point detection and description for event-based camera. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 5396-5405, 2023.
[40] Marco Cannici, Marco Ciccone, Andrea Romanoni, and Matteo Matteucci. Asynchronous convolutional networks for object detection in neuromorphic cameras. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 0-0, 2019.
[41] Yanxiang Wang, Xian Zhang, Yiran Shen, Bowen Du Guangrong Zhao, Lizhen Cui, and Hongkai Wen. Event-stream representation for human gaits identification using deep neural networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, (1733343333499 2021.
[42] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition scale. arXiv preprint arXiv:2010. / 1929, 2020.
[43] Wenhai Wang, Enze Xie, Xiang Li, Dengping Fan, Cheng Song, Ding Liang, Tong Lu, Ping Luo, and Ling Shao. Pyramid vision transformer: A versatile backbone for dense prediction without convolutions. Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 568-578.1 2021.
[44] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10012-10022, 2021.
[45] Haiping Wu, Bin Xiao, Noel Codella, Mengchen Liu, Xiyang Dai, Lu Yuan, and Lei Zhang. Cvt: Introducing convolutions to vision transformers. Proceedings of the IEEE/CVF International Conference Computer Vision, pages 22-31, 2021.
[46] Pierre De Tournemire, Davide Nitti, Etienne Perot, Davide Migliore, and Amos Sironi. A large scale event-based detection dataset for automotive. arXiv preprint arXiv:2001.08-4 2020.
[47] Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer VCCV 2014: 13th European Conference, Zurich, Switzerland, September 6- 12, 2014, Proceedings, Part V 13, pages 740-755. Springer, 2014.
13

---

[48] Matthias Fey and Jan Eric Lenssen. Fast graph representation learning with pytorch geometric. arXiv preprint arXiv: (tt.i1933002222 2019.
[49] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
[50] Leslie N Smith and Nicholay Topin. Super-convergence: Very fast training of neural networks using large learning rates. In Artificial intelligence and machine learning for multi-domain operations applications, volume 11006, pages 369-386. SPIE, 2019.
[51] Nikola Zubic, Mathias Gchrig, and Davide Scaramuzza. State space models for event cameras. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1819-58233 June 2024.
[52] Yansong Peng. Yueyi Zhang, Zhiwei Xiong, Xiaoyan Sun, and Feng Wu. Get: Group event transformer for event-based vision. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 6038-6048, October 2023.
[53] Nikola Zubić, Daniel Gehrig, Mathias Gehrig, and Davide Scaramuzza. From chaos comes order: Ordering event representations for object recognition and detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 12846-12856, October 2023.
[54] Yansong Peng. Yueyi Zhang, Peilin Xiao, Xiaoyan Sun, and Feng Wu. Better and faster: Adaptive event conversion for event-based object detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 2056-2064, 2023.
[55] Jianing Li, Siwei Dong, Zhaofei Yu, Yonghong Tian, and Tiejun Huang. Event-based vision enhanced: A joint detection framework in autonomous driving. In 2019 ieee international conference on multimedia and expo (icme). pages 1396-1401. IEEE, 2019.
[56) Dianze Li, Yonghong Tian, and Jianing Li. Sodformer: Streaming object detection with transformer using events and frames. IEEE Transactions on Pattern Analysis and Machine Intelligence,
2023.
[57] Daniel Gehrig and Davide Scaramuzza. Low-latency automotive vision with event cameras. Nature, 12344114)00 2024.
A Efficient VIT
This section extends section 3.5
by explaining the methodologies used in our architecture with Linear and is vital component of our model. For further details, readers
ViT, which originates from 24 are referred to the original paper.
Traditional ViTs not suitable for fast visual tasks due to their reliance extensive softmax attention. Softmax attention modules model interactions between every pair of tokens in the feature map aggregate spatial information, leading to high computational complexity. Efficient ViT is speedy visual model that replaces the computationally intense softmax calculations with multi-scale linear attention, maintaining excellent hardware efficiency while achieving global receptive field and multi-scale learning.
Firstly, to reduce computational complexity and hardware latency, softmax attention is replaced with linear attention, formulated as:
Sim (Qi, K2) V,
0,
(10)
Sim Sim(Q Q2, K,,)
where
Sim K) = ReLU(())(LL(())))
(1I)
14

---

This leads to the final output:
Re.U(Q,)) (E ReLU(K,)VV,
O, = ReLU((Q)) -j= ReLU((KyT))
(12)
significantly reducing both computational complexity and memory requirements.
However, due to the loss of local feature extraction capability by ReLUU, linear attention's 's performance lags behind softmax attention. To improve performance, two core solutions have been proposed:
1. Insert deep convolutions in each FFN layer. Linear attention extracts global features, while FFN+DWConv captures local information, minimizing computational overhead and greatly enhancing local feature extraction capability of linear attention.
2. By aggregating adjacent Q. K, V token information into multi-scale tokens, the multi-scale learning capability across different channels of linear attention is enhanced.
B Performance Scalability Analysis
To test the scalability of our models, we use inputs with three different event counts to train distinct weight parameters and conducted cross-testing under various input conditions. Given the similar performance outcomes of EGSST-B and EGSST-E, we present specific results only for EGSST-E.
Table 4: Performance scalability analysis with different number of input events. The results here are all run on the Gen1 dataset and the results on I Mpx are similar. (Note: T refers to thousand.)
EGSST-E (2T)
EGSST-E (10T)
EGSST-B (18T)
EGSST-E
Events
mAP (%) f
Time
mAP (%) 1
Time
mAP
Time
Params (M)
(ms) J
(ms) |
(%) f
(ms) |
2,000
34.9
4.6
39.4
4.6
37.7
4.6
12.3
10,000
33.1
6.1
49.6
6.0
44.5
6.2
12.3
18,000
30.6
7.9
45.4
7.9
51.7
7.8
12.3
In terms of runtime, it can be demonstrated that a higher number of events will invariably lead to a higher consumption of time. In terms of mAP, cross-test results demonstrate that model weights trained with fewer events perform well when applied to datasets with larger number of events, highlighting the excellent generalization capabilities our models. Conversely, although models trained with larger number of events show some performance disparity when applied to datasets with fewer events, the performance remains acceptable, indicating good adaptability of the model.
Our models allow for the flexible setting of the event count N, leveraging certain capabilities from dynamic graph processing the treatment of GCN. While a larger event count N introduces more node features, significantly enhancing detection accuracy, it also results in increased computational load and latency.
C Dynamic Label Augmentation
C.1 Methodology
In this paper, we propose novel dynamic label matching method to address the limitations of traditional label matching approaches when processing fixed numbers of event data. Specifically, the event data acquisition speed is extremely high, while existing datasets typically lack a sufficient number of labels to match the accumulated fixed-number event data. Therefore, traditional methods based on fixed time window label assignment are inadequate for scenarios requiring both adaptability to dynamic scenes and sufficient label coverage. To overcome this limitation, we design a Dynamic Label Augmentation approach to flexibly assign appropriate labels for each batch of fixed- number events.
15

---

In the proposed method, we assume that the fixed number of collected events is 10,000, for which labels need to be assigned. Suppose there are m labels in the original dataset, with each label correponning to a timestamp
For each batch of 10,000 events, we first compute the representative time, defined as the mean timestamp of these events, denoted by i,, where represents the index of the current batch of 10,000 events.
To achieve dynamic label matching, we introduce constant ? control the flexibility of the label matching time window. Specifically, we define a dynamically adjusted time range 7 At", where At" time parameter that varies in real-time based on the dynamics of the events. A label with a timestamp Tm is considered appropriate for the current batch of 10,000 events if it satisfies the following condition:
(13) Since At" changes dynamically according to the characteristics of the events, the proposed method effectively adapt to variations in the event data, ensuring the accuracy of label assignment.
f, - At" S Tm S ty. At
In summary, the Dynamic Label Augmentation method enables flexible adjustment of the time window, allowing the label assignment process to adapt to different dynamic properties of the events. This approach avoids potential mismatches associated with traditional fixed time window methods and exhibits strong robustness and adaptability.
C.2 Experiment
Table 5 iillsstrates the impact various data augmentation techniques on model accuracy, with dynamic label augmentation demonstrating relative advantage. Traditional augmentation methods, such as horizontal flipping, zooming in, and zooming out, improve model robustness by increasing data diversity, contributing accuracy improvements. However, dynamic label augmentation, which adaptively adjusts the label generation range, shows better adaptability when handling dynamic scenes, particularly by reducing the risk of label mismatches, thereby potentially enhancing overall model performance.
Table 5: Accuracy Improvement from Dynamic Label Augmentation. All augmentation tech- niques improve accuracy, with dynamic augmentation showing the greatest improvement.
h-fipp
zoom-in
zoom-out
dynamic
mAP (%)
38.3
41.8
43.3
42.1
46.6
49.6
The experimental results suggest that while conventional augmentation techniques are effective in improving accuracy, dynamic label augmentation may provide additional gains in both accuracy and robustness by flexibly adjusting the label generation process. This approach appears to improve the model's 's ability to adapt to complex and fast-changing environments to certain extent.
16

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The main claims the abstract and introduction accurately reflect the paper's contributions and scope because they succinctly summarize the key methodology employed in the research, providing a clear overview that aligns with the detailed content presented in the subsequent sections of the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
The paper discusses the limitations of the work performed by the authors in section
Justification:
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged
the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
17

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: This is an applied paper that provides an event- event-based processing framework,
and as such, it does not focus on theoretical results that require detailed assumptions and proofs.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
The paper fully discloses all the necessary information to reproduce the main experimental results by providing detailed descriptions of the methodologies and parameters used, ensuring that the main claims and conclusions can be independently reproduced.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
18

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification:
The paper provides open access the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in the supplemental material, and the code is included in the attachment.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please the NeurlPS code and data submission guidelines (https://nips.co
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
ssinnooiccoyy for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/eetails
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: This paper specifies all the training and test details necessary to understand the results section [
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to a level detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[No]
Justification: The experimental results in the paper do not include error bars or other information about the statistical significance of the experiments.
19

---

Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by enror bars, confi-
dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are range (e.g. negative етгог rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: The paper provides sufficient information on the computer resources needed to reproduce the experiments in section 4.2
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: The research conducted in the paper conforms to the NeurIPS Code of Ethics as it ensures the responsible use of data, respects participant privacy and commits to integrity and ethical standards in research.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
20

---

Question:
Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: There is societal impact of the work performed.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deploymeett of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: This paper poses such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
We cite the original papers or websites that produced the code package or dataset.
Justification:
Guidelines:
21

---

The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperssi thcode. com/ com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: This paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: This is an applied paper on event camera applications, and it does not involve crowdsourcing experiments research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
22

---

Justification:
The paper does not involve human experiments or study participants, so there are no potential risks describe, disclosures to subjects, and no need for IRB approvals.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
23


Reviews:
[{"review_id": "A1AfIXBtza", "reviewer": "Reviewer_KLe1", "summary": "The paper introduces a novel event-based graph spatiotemporal sensitive transformer framework aimed at enhancing the efficiency of object detection in dynamic vision systems. This framework leverages the unique properties of event camera data by modelling event data through a graph structure and incorporates key components such as the spatiotemporal sensitivity module and temporal activation controller to improve spatial and temporal processing efficiency. Additionally, the integration of a lightweight, multi-scale linear vision transformer further enhances processing efficiency.", "questions": "1.\tIt would be better for the manuscript to provide more detailed experimental validation and comparative analysis data to support the effectiveness and superiority of the framework, such as the Nighttime Driving Detection dataset.\n\n2.\tAccording to the content of reference [44] in the paper regarding AEC+DETR, there are factual errors. Please verify the specific metrics of this method on the two datasets and make the necessary corrections in the corresponding text.\n\n3.\tThe absence of visualized detection results on the dataset needs to be supplemented and enhanced.", "limitations": "See Weaknesses", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The introduction of the event-based graph spatiotemporal sensitive transformer framework, combining event data processing and graph neural network technology, brings new perspectives and methods to the field of object detection.\n\n2. By leveraging the unique properties of event camera data and incorporating key components such as the spatiotemporal sensitivity module, efficiency in spatial and temporal processing is significantly enhanced, leading to more effective object detection in dynamic vision systems.\n\n3. The integration of a lightweight, multi-scale linear vision transformer further boosts processing efficiency, enabling the framework to excel in handling large-scale continuous spatiotemporal data.", "weaknesses": "1.\tCould the manuscript clarify the advantages of your model over the AEC+YOLOv5 method mentioned in reference [44], and how the combination of GNN + LinearViT demonstrates its effectiveness?\n\n2.\tIt would be better to provide a concise visualization of the data to analyze whether the SSM and TAC modules genuinely simulate the natural prioritization of fast-moving objects within the field of view, while lowering the priority of slower objects, rather than merely presenting a seemingly reasonable narrative.\n\n3.\tPlease provide a concise analysis of the description concerning the comprehensive utilization of temporal information mentioned in the contributions. Moreover, are SSM and TAC critical steps that enhance the effective and precise focus on targets within the event data? Additionally, please clarify whether the Connected Subgraphs Construction is referenced for conducting ablation experiments to demonstrate its value.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your insightful feedback on our paper. We have revised the manuscript based on your suggestions and hope the changes meet your expectations. Please feel free to contact us with any questions or further information you might need during the review process.\n\n**W1.** Regarding your suggestion to compare our method with the AEC+YOLOv5[2] approach, we find it highly valuable and will include this comparison in Table 1. Due to page constraints, we have included additional experimental data in the author rebuttal for your reference. To ensure a fair evaluation, we have replaced the detection head from RT-DETR[4] to the YOLO series, and specifically, following another reviewer's advice, to YOLOX[5], which is more commonly utilized in event camera-based object detection. On the Gen1 dataset, although our EGSST-B-Y method exhibits slightly lower detection accuracy compared to the AEC+YOLOv5 method, it demonstrates a 5% improvement in inference speed and is notably more lightweight. Similarly, on the 1Mpx dataset, the results are consistent, with our method achieving more than double the inference speed of the AEC approach.\n\nIn the field of event data processing, we employ the Graph Transformer to effectively capture the spatiotemporal characteristics of event data using graph structures, while leveraging the Transformer's advantages in multi-scale information fusion and global feature extraction. Moreover, the linear nature of our model ensures high processing efficiency. As indicated by the comparisons in Table 1, although our model does not achieve the best performance on all evaluation metrics, its remarkably low inference time and parameter count demonstrate its effectiveness and significant contribution to the field of event data processing.\n\n**W2.** Thank you very much for the reminder that we have added an image of the visualisation results of SSM in the rebuttal pdf.\n\n**W3.** Thank you for your insightful questions and here are the relevant responses:\n\n1. **Utilization of Temporal Information in GNNs:** Our research utilizes Graph Neural Networks (GNNs) to effectively process spatiotemporal information, preserving the temporal attributes of event data throughout the transformation process. We employ an event-level transformation strategy for converting graph features to frame features, which significantly retains temporal information and boosts adaptability to dynamic environments.\n\n2. **Roles of the SSM and TAC Modules:** Our ablation studies have confirmed the pivotal roles of the SSM and TAC modules in focusing accurately on targets within event data. Removing the TAC module notably decreases precision, highlighting its essential role in enhancing detection accuracy.\n\n3. **Role of Connected Subgraphs:** The construction of connected subgraphs is crucial in filtering out noise and sharpening the model’s focus. Experiments show that processing 10,000 event points retains approximately 73% of events, which helps diminish noise interference. Eliminating smaller subgraphs allows the model to concentrate on those with critical labels, thereby boosting detection efficiency and accuracy. Visualizations in the SSM+TAC section demonstrate the distribution of these subgraphs effectively.\n\nWe appreciate your attention to these aspects of our work and hope these explanations address your queries satisfactorily.\n\n**Q1.** Thank you for your suggestions. Regarding the nighttime driving detection dataset, we find that the MVSEC-NIGHTL21 dataset [1,2] is an excellent choice. However, we regret not conducting experiments with this dataset due to time constraints and our analysis. Our method, along with the AEC[2] and ITS[3] methods, are event-based and do not rely on RGB frame information. Effective detection can be performed as long as there is sufficient event data. Furthermore, research on the paper [2] shows that while AEC and ITS experienced a decrease in detection accuracy on the MVSEC-NIGHTL21 dataset, they still maintained a high level of performance, demonstrating that event-based detectors are almost independent of RGB information.\n\n**Q2.** We are very sorry for this writing error and will make corrections in the table and the corresponding text in the paper.\n\n**Q3.** Thank you very much for your kind reminder. We have added the visualization results of our experiments on the relevant dataset. The results can be viewed in the rebuttal PDF.\n\n**References**:\n\n[1] The Multivehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception, 2018\n\n[2] Better and Faster: Adaptive Event Conversion for Event-Based Object Detection, AAAI 2023\n\n[3] Inceptive event time-surfaces for object classification using neuromorphic cameras, ICIAR 2019\n\n[4] Detrs beat yolos on real-time object detection, CVPR 2024.\n\n[5] Yolox: Exceeding yolo series in 2021, arXiv 2021.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Vlbmrp2nA1", "reviewer": "Reviewer_WqtW", "summary": "This paper uses graph structure to model event data and realize event classification. Spatiotemporal Sensitivity Module (SSM) and an adaptive Temporal Activation Controller (TAC) mimic the response of the human eyes in dynamic environments by selectively\nactivating the temporal attention mechanism based on the relative dynamics of event data, thereby effectively conserving computational resources. In addition, the integration of a lightweight, multi-scale Linear Vision Transformer (LViT) markedly enhances processing efficiency.", "questions": "The highlight of this paper is the improvement of efficiency. I want to know whether the time statistics of the algorithm include the Graph Construction process, and how to calculate the time of the algorithm? What percentage of the total process of the algorithm is occupied by the construction time? How do the relevant hyperparameters (such as the number of edges, neighborhood radius, etc.) affect the algorithm when used?", "limitations": "In addition to the limitations mentioned in Weaknesses, do the methods mentioned in this paper generalize to other visual tasks such as optical flow estimation? Verifying the generalization of the method proposed in this article will make the research more meaningful.", "rating": 5, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "In this paper, SSM and TAC, as well as multi-scale Linear Vision Transformer (LViT) are used to improve the efficiency of calculation while maintaining good classification performance. \nThe data augmentation method of Dynamic Label Augmentation is also mentioned as a potential contribution, but is lacking in specifics.", "weaknesses": "1. Insufficient experiment. The Gen1 and 1Mpx datasets used in this paper are not the most widely used datasets for event classification tasks. Even with a focus on driving scenarios, the N-CARS [HATS: Histograms of averaged time surfaces for robust event-based object classification] dataset should be included for comparison with other existing methods.\n2. The description of some details needs to be perfected. For example, the specific process of Dynamic Label Augmentation (312 lines) was not introduced, the key parameter $R$used in the experiment (137 lines, formula 2) was not clear, and the main idea of Efficient ViT (236 lines) used in the paper should also be briefly explained.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1.** Thank you for your attention to our choice of datasets. We acknowledge that N-CARS [7] is a significant dataset for event classification tasks. However, our research focuses on event-based object detection, which is distinct from event classification. Literature [1,2,3,4] supports our use of the Gen1 and 1Mpx datasets, which are well-established in object detection research, hence their selection for our studies. Nevertheless, we recognize the potential of the N-CARS dataset for future research in event classification. We plan to explore this dataset further and will discuss these plans in our paper, citing relevant literature to support our research direction and dataset choices.\n\n**W2.** Thank you very much for highlighting areas for improvement in our paper. We have provided more detailed explanations regarding Dynamic Label Augmentation, the key parameter R, and the Efficient ViT as follows:\n\n1. **Dynamic Label Augmentation:** We will detail the methodology in the appendix and summarize it here. This technique dynamically adjusts the size of label expansion windows based on the time span of accumulated events, enhancing label accuracy. Unlike traditional methods that use fixed ranges—potentially leading to errors in dynamic scenes—our method adapts the expansion windows to suit the speed of moving objects, minimizing incorrect labeling. For example, expansion windows contract for fast-moving objects to avoid mislabeling, and expand in slower-changing scenes to capture more potential labels. This adaptive approach is particularly effective in environments with variable dynamics, such as traffic monitoring or motion tracking, ensuring objects are labeled more precisely.\n2. **Parameter R:** We value your interest in the role of parameter R, a key hyperparameter determining the distance threshold for edges between graph nodes. We set R to 30 across two datasets, a value optimized through initial experiments to effectively construct graph data and generate adequate connected subgraphs for our analysis. We further enhance graph construction efficiency using the radius_graph function from the Pytorch Geometric library, which rapidly builds graphs based on this radius, thereby improving our model’s processing speed and efficiency.\n3. **Efficient ViT [5]:** We appreciate your interest in Efficient ViT. For comprehensive details, please see its original ICCV 2023 publication. In our appendix, we will outline the core design and technical merits of Efficient ViT for rapid visual tasks:\n   - **Softmax Replacement:** Traditional ViT models use softmax attention, which is effective but computationally intensive. Efficient ViT uses multi-scale linear attention to reduce computational complexity and hardware latency.\n     - $ O_i = \\sum_{j=1}^N \\frac{\\text{Sim}(Q_i, K_j)}{\\sum_{j=1}^N \\text{Sim}(Q_i, K_{j})} V_j  $\n     - $ \\text{Sim}(Q, K) = \\text{ReLU}(Q) \\text{ReLU}(K)^T $\n     - Consequently, $O_i=\\frac{\\text{ReLU}(Q_i) \\left(\\sum_{j=1}^N \\text{ReLU}(K_j) V_j\\right)}{\\text{ReLU}(Q_i) \\left(\\sum_{j=1}^N \\text{ReLU}(K_j)\\right)^T},$ drastically reducing computational complexity and memory demands.\n   - **Optimizing Local Feature Extraction:** Linear attention reduces computational demands but is less effective at capturing local details. We address this by including depthwise convolution (DWConv) in each Feed Forward Network (FFN) layer.\n   - **Aggregating Multi-Scale Token Information:** The model aggregates neighboring Q, K, and V tokens into multi-scale tokens, enhancing linear attention's ability to process data across different channels efficiently and accurately.\n\n**Questions:**\n\n**Re.** Thank you for your detailed inquiry about our model. We are happy to provide clarifications:\n\n1) Time Statistics: Our algorithm's timing involves the graph processing module, the Linear ViT module, and the detection head module. Graph processing includes constructing the graph and operations within the Graph Neural Network (GNN) and Spatiotemporal Sensitivity Module (SSM), with precise GPU timing provided by PyTorch's torch.cuda.Event.\n\n2) Time Proportion: The graph processing module, utilizing the Pytorch Geometric library and the torch_scatter library for efficient sparse operations, is highly efficient, accounting for approximately 31.4% of the total algorithm time. This efficiency significantly boosts our model's performance in dynamic environments.\n\n**Limitations:**\n\nThank you for highlighting the potential applications and emphasizing the need for broader validation of our method. Currently, our research focuses on event-based object detection, allowing us to refine detection techniques specific to event data.\n\nHowever, we understand the importance of extending these techniques to a wider range of visual tasks. In future work, we plan to test the model's applicability to other areas, including event classification [1, 2, 4, 6] and optical flow estimation. We will continue our research to evaluate the generalization and adaptability of these methods and report our findings. We look forward to our continued contributions to both the research and practical applications in event camera vision.\n\n**References:**\n\n[1] Learning to detect objects with a 1 megapixel event camera, Advances in Neural Information Processing Systems, 2020.\n\n[2] Recurrent vision transformers for object detection with event cameras, CVPR 2022.\n\n[3] Asynchronous spatio-temporal memory network for continuous event-based object detection, IEEE Transactions on Image Processing, 2022.\n\n[4] Better and faster: Adaptive event conversion for event-based object detection, AAAI 2023.\n\n[5] Efficientvit: Lightweight multi-scale attention for high-resolution dense prediction, ICCV 2023.\n\n[6] Aegnn: Asynchronous event-based graph neural networks, CVPR 2022.\n\n[7] HATS: Histograms of averaged time surfaces for robust event-based object classification, CVPR 2018.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zboqDL2uxs", "reviewer": "Reviewer_NWmb", "summary": "The paper proposes a novel event-based graph spatiotemporal sensitive network (EGSST), which is the first work using Graph Transformer for object detection tasks on event cameras. This work primarily involves two key innovative modules: a spatiotemporal sensitivity module (SSM) and an adaptive temporal activation controller (TAC). Additionally, the integration of a lightweight, multi-scale Linear Vision Transformer (LViT) significantly enhances processing efficiency. The results demonstrate that the proposed EGSST achieves state-of-the-art (SOTA) performance, especially outperforming other graph neural networks for event-based object detection.", "questions": "Please see the weakness and response each comment. Besides, Additionally, the authors should also answer the following question: Could the authors plan to further expand this unimodal work to multimodal work by integrating GNN for event processing and Transformer for image processing in the future?\n\nIf so, please cite some multimodal object detection methods [1, 2, 3] using events and frames.\n\n[1] Event-based vision enhanced: A joint detection framework in autonomous driving, ICME 2019.\n\n[2] SODFormer: Streaming object detection with transformer using events and frames, TPAMI 2023.\n\n[3] Low-latency automotive vision with event cameras, Nature 2024.", "limitations": "A minor flaw is that the authors claim to be the first to apply Graph Transformer to event-based object detection, seemingly by merely linking feature maps generated by GNNs directly to DETR. It suggests that the authors revise a statement to be slightly more conservative.", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "i) Graph neural networks for event-based object detection is a promising solution to achieve low-latency in real-world applications.\n\nii) The proposed EGSST achieves SOTA performance, especially outperforming other GNNs for event-based object detection.\n\niii) The writing is straightforward, clear, and easy to understand.", "weaknesses": "i) The authors claim that the first Graph Transformer worked on event-based object detection, seemingly simply linking the feature maps generated by GNNs directly to DETR. Of course, I am certain that the author has contributed to the handling of events in spatiotemporal GNNs. However, the author may not make any innovative work on the unique combination of GNNs and Transformer. Could the author clarify the innovative aspects of the combination of graph transformer and others?\n\nii) In Table 1, the authors should separately place the existing method of event-based object detection using GNNs in the bottom few rows of the table. There are still several works that have not been searched and cited, such as DAGr in Nature 2024. In addition, by comparing the GNNs method, it can be found that the proposed EGSST has good performance and inference time.\n\niii) The authors' use of DETR to regress the output of object detection results is not consistent with the YOLOX approach used by existing GNNs for event-based object detection. It is challenging to determine whether the DETR module has significantly improved performance. The authors should conduct an experiment with a YOLOX detection head to clarify this.\n\niv) In the ablation experiment, the authors should ablate important parameters in the proposed innovative modules (i.e., SSM and TAC). In addition, the expansion experiment on the accumulation of event windows in Table 5 takes up too much space, which may affect the length of other important experimental content.\n\nv) The writing section needs further improvement. For example:\n\na. In related works, the author can write according to event-based object detection, graph neural networks for event data. This writing may better reflect the focus and novelty of the work.\n\nb. The biggest advantage of GNNs in processing event data is low latency. The author proposes a method that basically reaches the millimeter level, which is very good. It is recommended to emphasize the low latency advantage of the method more.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1.** We greatly appreciate your evaluation and suggestions regarding our integration of Graph and Transformer technologies. In our integrated framework, there are two key aspects:\n\n1) Interaction Mechanism between SSM and TAC: The Spatiotemporal Sensitivity Module (SSM) we designed processes event data based on the graph structure and directly influences the activation state of the Temporal Activation Controller (TAC). This design enables the TAC to precisely focus on the temporal dynamics of the event data, thereby enhancing processing efficiency and response speed.\n\n2) Event-level Information Transformation: In converting graph features to the format required by Transformers, we employ event-level operations. By accumulating features of each event at their corresponding spatial locations, each location not only retains rich event information but also reflects underlying dynamic changes, which is critical for object detection in dynamic vision systems.\n\nOf course, we acknowledge the shortcomings pointed out by you in this section and in the later Limitations. Therefore, we have attempted to make our statements about the Graph Transformer more conservative, as follows:\n\n- \"Our model effectively combines the advantages of Graph and Transformer technologies. Based on event data, it is lightweight, fast, and accurate, providing a novel technological approach for performing object detection tasks within event data.\"\n\n\n\n**W2.** Thank you very much for your careful reading and valuable suggestions. We will reorganize and optimize the method ordering in Table 1 according to your suggestions and will include some of the latest related literature, including S4D-ViT-B[1], S5-ViT-B[1], GET-T[2], and ERGO-12[3], to ensure our paper covers the most current research advancements.\n\nFurthermore, based on your recommendation [4], we learned about the DAGr method, an innovative hybrid approach that combines event cameras with traditional frame cameras. DAGr leverages the high temporal resolution and sparsity benefits of event cameras, along with the rich contextual information from frame cameras, to achieve efficient, fast object detection while significantly reducing perception and computation latencies. Unfortunately, the DAGr method was published online just days after the NeurIPS 2024 submission deadline, which is why we were unable to reference this excellent work in time. However, we are impressed by this method and plan to use it as an important benchmark in our future multimodal research. \n\nReferences:\n\n[1] State Space Models for Event Cameras, CVPR 2024\n\n[2] Get: Group event transformer for event-based vision, ICCV 2023\n\n[3] From chaos comes order: Ordering event representations for object recognition and detection, ICCV 2023.\n\n[4] Low-latency automotive vision with event cameras, Nature 2024.\n\n\n**W3.** Thank you for your detailed attention to our choice of detection head technology. In response to your concerns about using the DETR or YOLO series detection heads, we have added additional experimental results using YOLOX as the detection head. These new results will be included in Table 1 to visually demonstrate the performance of YOLOX within our framework.\n\nDue to the word count and PDF page limitations of the rebuttal, we are unable to detail all of the revisions and comparisons here. Therefore, only the additions in Table 1 are given in the author rebuttal.\n\n\n\n**W4.** Thank you for your detailed guidance on our ablation study design and the layout of our paper. To evaluate the impact of SSM and TAC modules, we add ablation experiments such as $\\pi$ in SSM and Fst in TAC. The experimental results indicate that while $\\pi$ has a negligible overall effect, generating Q from Fst with TAC always active improves precision, increasing mAP accuracy by 0.3% with only a 0.01 ms increase in time delay and no additional parameter increase. More detailed ablation experiments will be added in the Appendix. Additionally, we have included visualizations of SSM in dynamic environments, further confirming the effectiveness of these combined modules. These results will help readers more intuitively understand the roles of the modules.\nRegarding your concern about the space occupied by the extended experiments on event window accumulation, we have recognized this issue and plan to move this section to the appendix in the revised manuscript to save space in the main text and maintain the compactness of the paper.\n\n**W5.** Thank you very much for your specific suggestions regarding our writing. We will carefully revise and improve our paper based on your feedback.\n\n1) Related Work Section: We will reorganize the \"Related Work\" section to more detailedly categorize and discuss it according to event-driven object detection and graph neural network processing of event data, in order to better highlight the focus and innovations of our work.\n\n2) Emphasizing Low Latency Advantage: We will more clearly emphasize in the paper the achievements of our method in achieving millisecond-level latency.\n\nDue to word count limitations, we will reserve more detailed modifications and explanations for the revised manuscript. We hope these improvements will fully meet your expectations and further enhance the quality of the paper. Thank you for your valuable suggestions, and we look forward to you seeing these changes in the revised manuscript.\n\n**Questions:**\n\nThank you for your question. Indeed, in future research, we plan to expand our current unimodal work to multimodal efforts. We believe that by integrating data from event cameras and traditional frame cameras, we can further enhance the system's perceptual capabilities and response speed, especially in dynamic and complex visual environments.\n\n To support our future research direction and acknowledge the existing work in our field, we will cite the excellent studies you mentioned in the \"Discussion and Conclusion\" section of our paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8zGa2fQNdA", "reviewer": "Reviewer_NBbr", "summary": "This paper introduces a novel event-based object detection network by processing event-based data as graph data. After incorporating an SSM that acts as the selection role, graph convolution neural networks, and various attention modules, the proposed pipeline achieves good results while retaining high efficiency. Ablation studies prove the effectiveness of the proposed modules. Although the overall results are outstanding, the overall writing is poor, making the paper hard to follow.", "questions": "1. In line 294, the author indicates that the parameters involved in the forward computation vary dynamically due to the SSM module. Could the authors provide some statistical and qualitative results on the ratio of involved parameters to help readers better understand the role of SSM?\n\n2. How is the gating signal produced from $F_{st}$ (line 215)? Could the author provide experimental results to prove the effectiveness of generating $Q$ from $F_{st}$ instead of $\\mathcal{F}$?\n\n3. In this paper, only quantitative results are shown, making me doubt whether I am reading a paper in the CV field, especially for the object detection task. Could the authors show more qualitative results? I think this is more valuable and significant than the data augmentation results which can be moved to the appendix.", "limitations": "The authors are very transparent about their limitations on deploying the GNN into production.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. Processing event-based data as graph data to retain its spatiotemporal information is an interesting direction worth exploring.\n2. The novel design of the SSM module suits the property of event-based data well.\n3. The experiments illustrate the effectiveness and efficiency of the proposed method.", "weaknesses": "1. In section 3.3, the author claims that the SSM module mimics the human eye to output the degree of object dynamics in the event stream. This metric is measured by the number of events and the time span of each subgraph since more events usually indicate faster object movement. However, this measurement does not consider huge objects in the static background which also generate a large number of events with relatively low speed, which does not align well with the author's claim. Also, the definition of $f(\\cdot)$ of SSM is not clear, which is only given by the author in line 176 as an example.\n\n2. The $\\pi$ function in section 3.3 is not detailedly introduced either.\n\n3. In line 302, the author mentions that the CNN module is removable. Where is this removable CNN module? Additionally, the TAC module, Linear ViT, and DETR all process frame-based data, and CNN is included in the Linear ViT as illustrated in Figure 2. Based on these observations, I don't think the framework is event-based.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your constructive feedback. We have carefully revised the manuscript based on your suggestions, particularly regarding the clarity of function definitions and the roles of specific modules. We hope these revisions meet your expectations and ask for your reconsideration during the review process. We are committed to addressing any further questions or requirements during the discussion period.\n\n**W1.** In response to the first issue you raised in the weaknesses section, we provide the following detailed replies and corrections:\n\n1) To further illustrate the relationship between object and background dynamics, we have added an image of SSM in the rebuttal PDF, positioned near section 3.3 of the manuscript. Regarding the dynamics, large objects generate a large number of events, but their relatively slow movement speed results in a large time span. These events significantly influence the computation of the Event Global Motion (EGM), effectively reflecting the overall scene dynamics. Conversely, fast-moving objects, while generating a similar number of events, have a shorter time span, leading to a higher Event Local Motion (ELM).\n\n2) We originally designed $f(\\cdot)$ to quantify the dynamics in the event stream using the number of events and time intervals. However, our experiments indicate that a simple proportional function $f(x, y) = \\frac{x}{y}$ adequately captures these dynamics without needing additional parameters from neural networks. To clarify and prevent misunderstandings, we will refine our explanation of this function in the revised manuscript, moving beyond examples to a clear definition.\n\n**W2.** Thank you for highlighting the unclear definition of function $\\pi$ in our manuscript. To address this, we will enhance our description in the revised paper. Function $\\pi$ performs aggregation operations—minimum, maximum, and mean—to enable lightweight processing with flexibility. In our setup, we primarily use the mean method to compute centroid coordinates of input data points. Our findings suggest that the choice of aggregation method does not significantly affect the overall detection performance. We will detail this analysis in the revised manuscript, providing ablation study results to clarify the effectiveness of function $\\pi$.\n\n**W3.** Thank you for your comments.\n1) To avoid any confusion about the CNN module, we will clearly mark this removable component in the updated Figure 2. \n2) Regarding data processing being event-based or frame-based: Our method incorporates components from established object detection frameworks like YOLO and DETR but differs fundamentally in data processing and feature extraction. Initially, our feature extraction relies solely on graph structures to process event data, preserving rich spatiotemporal information unlike traditional methods that compress events into frames. Secondly, when converting graph features to frame features, we implement event-level operations. By mapping each event's features to specific spatial locations, we ensure that every position within the frame contains detailed event-level information. This approach maintains the integrity of the event data throughout the frame-based output, significantly reducing the information loss typical of standard frame compression techniques.\n\n**Q1.** In response to the first question, we recognize the need for more detailed explanations in our manuscript and offer the following clarifications:\n\n1) Experimental Setup: We used a fixed batch size of 1 across 30,000 batches, activating the TAC module for about 34.4% of the total event data. These details will be included in section 4.4 to better illustrate the dynamics of the TAC module during operations.\n\n2) Correction of Errors: We identified errors in our previous ablation studies setup where the shuffle option was incorrectly active, and the batch size was not consistently 1. This affected the numerical results in the TAC Inactive line, and the corrected mAP should be 42.1%, with processing time of 4.3 ms. These corrections will be detailed in the revised manuscript.\n\n**Q2.** Thank you for your attention to the details of our research methodology.\n\n1) Flowchart Supplement: We have detailed the process flowchart of the TAC module in the revised Figure 2, which can be viewed in the PDF file.\n\n2) Ablation Study Results: To verify the effectiveness of generating Q from Fst, we designed an ablation study to compare the outcomes of generating Q from Fst and from F. The results show that the method of generating Q from Fst with TAC always active increased the mAP accuracy by 0.3%, while the time delay increased by only 0.01 ms and did not lead to any additional parameter increase. Although there is a slight time delay, considering that the improvement in accuracy is more significant compared to the minimal time delay, we believe that obtaining Q from Fst is an effective and worthwhile method.\n\n**Q3.** Thank you for your feedback. Based on your suggestion to include more qualitative results, we have implemented the following improvements:\n1) SSM Visualization: We've added visualizations of the SSM module's output to the PDF, providing a clear view of object dynamics within scenes.\n2) Qualitative Detection Results: We've incorporated several images showing detection outcomes in various real-world settings, highlighting our model's effectiveness in complex environments.\n3) Data Augmentation Details: As suggested, we will move the data augmentation results to the appendix, focusing the main text more on our key contributions.\n\n**Limitations：**\n\nDeploying GNNs into production involves addressing practical challenges such as computational resources, inference performance, and data storage and management, requiring concerted efforts in both hardware and software.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank all reviewers for their very informative feedback. We have responded separately to each reviewer and attached a PDF file with figures and tables to enhance our rebuttal. Additionally, considering the page limitations of the PDF, we have briefly presented the new experimental data from Table 1 here for clarity. More detailed modifications will be reflected in the final manuscript. We appreciate all feedback from the reviewers and will incorporate it into the final version of our manuscript.

The additional experiments in Table 1 are listed below:

| Methods                | Dataset     | mAP (%)     | Time (ms) | Params (M) |
| ---------------------- | ----------- | ----------- | --------- | ---------- |
| EGSST-B                | Gen1 / 1Mpx | 44.6 / 45.4 | 4.6 / 5.1 | 3.5        |
| EGSST-E                | Gen1 / 1Mpx | 49.6 / 50.2 | 6.0 / 6.3 | 12.3       |
| EGSST-B-Y (with YOLOX) | Gen1 / 1Mpx | 43.9 / 44.1 | 3.7 / 5.0 | 2.9        |
| EGSST-E-Y (with YOLOX) | Gen1 / 1Mpx | 47.8 / 48.4 | 4.2 / 5.3 | 10.4       |

Meta-review: This work proposes a novel event-based graph spatiotemporal transformer for object detection. It achieves SOTA performance with low inference time.

The strengths of this work are listed as follows.

1. Graph Neural networks for event-based object detection are novel and promising.
2. The spatiotemporal sensitive module (SSM) mimics the behavior of the human eye, which is suitable for event-based data. 
3. The proposed method achieves SOTA in event-based object-detection tasks with low inference time.

The weaknesses of this work are listed as follows.

1. Dynamic Label Augmentation seems to be an important part of the proposed approach, but the authors do not provide its details. The details and ablation study about it should be conducted. The authors mentioned it in the rebuttal; please provide more details in the new version of this work. 

2. In this work, an aggregation function is used to alleviate the burden of data processing. An ablation study should be conducted to evaluate its impact. It is unclear where the proposed method improves time efficiency. The authors refering the computation cost to another paper, it would be better to include such study in this work.

3. This work claims to be the first work to apply GNN to event-based object detection. However, it merely links feature maps generated by GNN to DETR. The authors stated in the rebuttal that they would revise such a claim.

Please take into account the reviews for the camera-ready version. Especially giving more details about each module/function used in this work, conducting necessary experiments to evaluate the impact of the aggregation function, soften the claims, etc.
Decision: Accept (poster)