Rethinking Scale-Aware Temporal Encoding for Event-based Object Detection
Authors: Lin Zhu
LongTengyu
Xiao Wang
Lizhi Wang
Hua Huang
Venue: NeurIPS 2025 poster ()
Keywords: Event-based object detection
temporal feature extraction
Primary area: applications

Abstract: Event cameras provide asynchronous, low-latency, and high-dynamic-range visual signals, making them ideal for real-time perception tasks such as object detection. However, effectively modeling the temporal dynamics of event streams remains a core challenge. Most existing methods follow frame-based detection paradigms, applying temporal modules only at high-level features, which limits early-stage temporal modeling. Transformer-based approaches introduce global attention to capture long-range dependencies, but often add unnecessary complexity and overlook fine-grained temporal cues. In this paper, we propose a CNN-RNN hybrid framework that rethinks temporal modeling for event-based object detection. Our approach is based on two key insights: (1) introducing recurrent modules at lower spatial scales to preserve detailed temporal information where events are most dense, and (2) utilizing Decoupled Deformable-enhanced Recurrent Layers specifically designed according to the inherent motion characteristics of event cameras to extract multiple spatiotemporal features, and performing independent downsampling at multiple spatiotemporal scales to enable flexible, scale-aware representation learning. These multi-scale features are then fused via a feature pyramid network to produce robust detection outputs. Experiments on Gen1, 1 Mpx and eTram dataset demonstrate that our approach achieves superior accuracy over recent transformer-based models, highlighting the importance of precise temporal feature extraction in early stages. This work offers a new perspective on designing architectures for event-driven vision beyond attention-centric paradigms. Code: https://github.com/BIT-Vision/SATE.

Rethinking Scale-Aware Temporal Encoding for Event-based Object Detection
Lin Zhu', Tengyu Long', Xiao Wang", Lizhi Wang", Hua Huang
School of Computer Science, Beijing Institute of Technology
2 School of Computer Science and Technology. Anhui University
School of Artificial Intelligence, Beijing Normal University linzhu @ bit.edu.cn, longtengyud @ bit.edu.cn xiaowang ahu.edu.en, wanglizhi www..edu.cc.......omoeeemeedb edu.cn, huahuang @ bnu.edu.cn
Abstract
Event cameras provide asynchronous, low-latency, and high-dynamic-range visual signals, making them ideal for real-time perception tasks such as object detection. However, effectively modeling the temporal dynamics of event streams remains a core challenge. Most existing methods follow Trame-based detection paradigms, applying temporal modules only at high-level features, which limits carly-stage temporal modeling. Transformer-basee based approaches introduce global attention to capture long-range dependencies, but often add unnecessary complexity and overlook fine-grained temporal cues. In this paper, propose CNN-RNN hybrid framework that rethinks temporal modeling for event-based object detection. Our approach is based
key insights: (1) introducing recurrent modules lower spatial scales to preserve detailed temporal information where events are most dense, and (2) utilizing Decoupled Deformable-enhaneed Recurrent Layers specifically designed according to the inherent motion characteristics of event cameras extract multiple spatiotemporal features, and performing independent downsampling at multiple spatiotemporal scales to enable flexible, scale-aware representation learning. These multi-scale features are then fused via a feature pyramid network to produce robust detection outputs. Experiments on Gen1, Mpx and eTram dataset demonstrate that our approach achieves superior accuracy over recent transformer-based models, highlighting the importance of precise temporal feature extraction early stages. This work offers a new perspective on designing architectures for event-driven vision beyond attention-centric paradigms.
Code: https://gitthu..co//BBBB/
1 Introduction
Event cameras provide fundamentally different visual sensing modality by asynchronously recording pixel-level brightness changes with microsecond latency [8] Their high dynamic range, sparse output, and low power consumption make them well-suited for real-time perception in high-speed or low-light environments [46) These characteristics render event cameras particularly attractive for performing visual tasks such as object detection 27 31. tracking 9 39. and optical flow estimation 44 6, 42
even challenging scenarios (c.g., extreme lighting variations and high-speed motion dynamics).
Event data is stored as asynchronous arrays containing the spatial coordinates, polarity, and timestamp of each illumination change. In contrast, conventional frame-based cameras represent visual infor- mation through fixed-rate pixel value matrices. This fundamental discrepancy renders conventional
"Coreepponiigg author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Fanel abrem
Eivent stheam
Event
Festure Viundlationn
Fecture Vararization
Feature Visual
Pacare
Recument
Paccment
Resummnt
Hesd
Erisnced
Blokk
Mack
Festure
Exadar
Ditaction
Detection Head
Detection Head
(a) High-Level Parallel RNN-based network
(b) High-Lavel Cascaded RNIN-based network
[c)
Figure 1: Comparison of different temporal modeling strategies in event-based object detection. Specifically, (a) adopts a High-Level Parallel RNN-based network (e.g.. DMANet (38) where RNN modules are independently inserted at different feature scales. (b) shows a High-Level Cascaded RNN-based network (e.g.. RED RVT M.. SAST 121), in which RNN modules are stacked sequentially across multiple scales to achieve cascaded temporal aggregation. In contrast, (c) il- lustrates Ours, where enhanced recurrent modules are inserted before significant downsampling. Multi-scale features are extracted separately via scale-specific branches. Here, RNN Block refers to basic recurrent units such as ConvLSTM [33, while Recurrent Layer combines recurrent modeling with additional feature extraction operations. The proposed Enhanced Recurrent Layer design is detailed in Sec. 3.
image-based neural networks incompatible with event data processing. To address this issue, existing methodologies are primarily categorized into two methodological branches. The first approach involves transforming raw spatiotemporal event streams into dense representations analogous to multi-channel images (such voxel grid [45]. event histogram [28]. and time surface [20 34
Such transformations facilitate the application of established computer vision techniques originally designed for frame-based data. The second category adopts sparse computational paradigms, typi- cally employing spiking neural networks (SNNs) or graph-based architectures, yet these frequently encounter challenges including hardware incompatibility and suboptimal accuracy. In this work, we utilize dense representations for their advantages in effectiveness.
In addition, how to effectively modeling the temporal dynamics of sparse event streams remains an open challenge, particularly for dense prediction tasks such as object detection. Recent approaches for event- based object detection span a diverse range of architectures. Transformer- based meth- ods [TI 30 leverage global attention to capture long-range temporal dependencies but are often computationally expensive and less effective at modeling localized, high-frequency temporal patterns. exploit the asynchronous nature of events but suffer from limited representation capacity and optimization difficulty. More recently, state-space models [50, 40) such Mamba have shown potential for sequential modeling but remain underexplored in the context of low-level event representation. Notably, many of these methods emphasize temporal modeling at deeper layers (Figure D), where spatial resolution is low and temporal cues may already be degraded.
Spiking neural networks (SNNs) 4 41 47
In this work, we argue that accurate modeling of early-stage temporal features is essential for effective event-based object detection. Since events are inherently sparse and localized in both space and time, the most informative temporal structures often appear low-level representations. Motivated by this, we propose to revisit convolutional architectures and augment them with effective recurrent modules at early stages to explicitly preserve fine-grained temporal information. Our design is not aimed at merely reducing model complexity, but at improving temporal fidelity in the feature extraction process.
To this end, we introduce a CNN-RNN hybrid network tailored for event-based object detection. The network consists of three key components. First, we place recurrent blocks at lower spatial scales to enhance temporal modeling at early stages, enabling the network to capture fine-grand fine-grained temporal patterns that are critical for understanding sparse event dynamics. Second, we propose a sonidd--nd-coquuee methodology that enhances current event features through decoupled per- pixel motion estimation and spatiotemporal feature fusion. Third, we design a multi-branch backbone with independent temporal downsampling at three resolution levels, allowing flexible and complementary spatiotemporal feature extraction across varying receptive fields. The multiple spatiotemporal features extracted by the backbone then are hierarchically propagated to detection head incorporating

---

Feature Pyramid Network (24), leveraging multi-scale features for accurate and temporally con- sistent object predictions. Despite its architectural simplicity, our model achieves ofte-of-thhearr performance on the Genl dataset, outperforming recent transformer- and SNN-based baselines.
In summary, the contributions of this work are:
(I) We propose a CNN-RNN hybrid architecture for event-based object detection that rethinks temporal modeling by introducing recurrent modules at lower spatial scales, enabling the capture of fine-grained temporal dynamics from sparse event streams where information is most dense.
(2) We propose strategic utilization of temporal information, that decouples the estimation of per-pixel motion from the feature fusion. Meanwhile, we design a scale-specific spatiotemporal encoding strategy that performs independent temporal downsampling across three resolution branches, allowing the network to flexibly extract complementary multi-scale features, which are later fused via Feature Pyramid Network for robust detection.
aediid--nd-oonquue methodology that enhances current event features through
(3) Extensive experiments on the Genl, IMpx and e'Tram benchmark demonstrate that our approach achieves state-of-the-art performance, outperforming recent transformer- and SNN-based meth- ods, validating the effectiveness of early-stage temporal modeling.
2 Related Works
Existing event-based object detection methodologies can be broadly categorized into two major branches: sparsity-aware aware architectures and dense feedforward networks.
Sparsity-aware architectures leverage bio-inspired neural designs, notably graph neural networks (GNNs) and spiking neural networks (SNNs), to directly process asynchronous event streams. GNN-based approaches dynamically construct spatiotemporal graphs through event subsampling strategies [32 10]. identifying temporally and spatially proximate nodes and establishing adaptive edge connections. A key challenge is designing graph topologies that enable efficient information propagation across long spatiotemporal ranges while remaining computationally tractable. SNNs, on the other hand, exploit sparse, time-dependent spike-based computation 44 [ inherently aligning with the event-driven nature of the data. Like RNNs, SNN neurons maintain temporal states, but emit spikes only when membrane potentials exceed a threshold, introducing con-differentiabiiity into the network. While surrogate gradients (26) can circumvent this issue, they often compromise the sparsity advantage. Despite their theoretical suitability, both GNN- and SNN- methods face practical limitations in terms of hardware dependency and inferior detection accuracy compared to dense alternatives.
Dense feedforward architectures represent the second major branch. Early methods convert event streams into fixed-duration frame representations and apply standard image-based detectors each temporal slice independently [3] [6 18 15). These methods largely ignore temporal continuity and motion cues, making them less effective in cases involving occlusion, low texture, or slow motion. Later approaches integrate CNNs with RNNs [3] 22 38), combining the spatial efficiency of CNNs with the sequential modeling capability of RNNs, yielding significant performance gains. architectures have gained prominence in event-based detection as well, leveraging self-attention to model long-range depen- dencies. Specifically, RVT [1]] employs recurrent vision transformer that integrates local-global self-attention with lightweight LSTM [ bbased temporal aggregation for efficient spatiotemporal modeling, while GET (30 introduces grouped token mechanisms and dual self-attention to jointly capture spatial, temporal, and polarity cues for enhanced event representation. In contrast, SSM [50] replaces recurrent operations with continuous-tine state-space modeling 12 355, enabling efficient and frequency- formers often suffer from computational inefficiency due to the sparsity and noise in event data. Efforts such as SAST (2 attempt to mitigate this by pruning irrelevant tokens and windows via attention-basee mechanisms. However, these strategies inevitably compromise the model's global receptive field, limiting their effectiveness in complex scenes.
With the advent of the Vision Transformer (Vi transformer-baseee
temporal aggregation across varying inference rates. While effective, trans-
Our investigation reveals that existing CNN- and Transformer-bases approaches combined with recurrent modules primarily emphasize high-level temporal modeling. However, due to the inherent sparsity of event data, reducing spatial resolution often results in the loss of crucial details essential
3

---

fg and
ng and
Ag and
Defermable enttanced Recument
Deformble- onhanced Recument Layer
Deformblle enhanced Recurent
Downsample
FPN+Htead
Leyer
Layer
Downsample
Downsamplo
bi-n-1 and 42-n-
hy and 6i
Ng and oi
Bll
Decoupled Deformable- enhanced Recurentt Layer
Decoupled Defurmblee enhanced Recurent Layer
Defemabiee enhenced Recumrnn
Downsample
Delection
Layer
Downsample
ptopapation
Temporal propogadonn
Downsample
Figure 2: Overview of our designed CNN-RNN network. The event data is first converted into tensor representation before being fed into the first stage. After passing through a shared shallow stem, the data processed by our Decoupled Deformable-enhance Recurrent Layer for spatiotemporal feature extraction. The extracted spatiotemporal features are then further processed in a more flexible manner. Finally, the resulting multi-scall features are fed into the detection head.
for accurate detection. To address this limitation, we strengthen early-stage temporal modeling by integrating event-specific recurrent modules at lower spatial scales, where event activity is denser and temporal cues are richer. Moreover, we introduce multi-branch downsampling scheme to refine multi-scale spatiotemporal representations, enabling flexible retention of critical information while mitigating information degradation.
Method
This section presents the proposed CNN-RNN hybrid architecture for event-based object detection. Our design is motivated by two core observations: (1) fine- grained temporal patterns in event data often appear at early stages and are best captured before spatial abstraction; and (2) scale-specific spatiotemporal encoding enables complementary feature extraction across object sizes and motion dynamics. Meanwhile, motion estimation, further leveraging temporal information, which not only enhances the current frame features, but also leverages the smoothing effect of deformable convolution effectively suppress the propagation of task-irrelevant information in low-dimensional features. The overall framework is shown Figure]
adopt divide-and-conquee approach to decouple feature fusion from
3.1 Event Representation
Event cameras output a stream of asynchronous events represented as tuples (x. (,p), where (3r, y) denotes the spatial location, í is the timestamp, and {+1, is the polarity indicating intensity increase or decrease. To make the data compatible with convolutional processing, we discretize the event stream into voxel grids (45) over a short temporal window. The events within temporal window AT' are converted into a voxel grid of size B x H x W, where H and W denote the height and width of the event frame, respectively, and B represents the number temporal bins.
(o,, Σ piòa(æ (x x1)60(1 y: 0z (7),
where t B-1 (t, -1) and da(a) = mar 1- (al). Our models use a time window AT 50mm AT
and B = 5 temporal bins.
3.2 Overall Architecture
The proposed model consists of three major components: Early-stage recurrent encoding for fine- grained temporal modeling, Decoupled Deformable-enhance Recurrent Layer (DDRL) designed

---

based on the divide-and-conqeen principle and the inherent motion information of event data, Scale- specific spatiotemporal branches with independent downsampling.
As shown in Figure 2 the input voxel slice is processed through a shared shallow stem (stride = 1) followed by three successive Decoupled Deformable-enhanced Recurrent Layers with built-in downsampling, yielding spatiotemporal features at three different scales. These features are then fed into three parallel branches, each employing its own temporal encoding mechanism a specific spatial scale. It is worth noting that each 2x downsampling is accompanied by a doubling of the channel dimensions, except for the final downsampling each branch, which is intended reduce the overall model complexity. The outputs of all branches are subsequently passed to the detection head for multi-scale object detection.
3.3 Temporal Modeling at Lower Scales
A key challenge in object detection with event cameras lies in whether neural networks can effectively extract meaningful information from both recent events and those generated several seconds earlier. Since event cameras respond to intensity changes caused by object motion, they produce very few events when objects move slowly or remain stationary. This inherent sparsity of event data means that critical information is often embedded in subtle details, which becomes particularly crucial when performing detection across multiple consecutive frames. To preserve this critical temporal information, we place the proposed Decoupled Deformable-enhance detailed in Section 3.4) at higher-resolution stages with downsampling factors of 2, 4, and 8. To effectively capture both short-term and long-term dependencies, our model also processes temporal sequences at multiple spatial resolutions. However, unlike previous works B1 38 II emphasizing high-level temporal information propagation, introduce temporal recurrence before significant spatial downsampling, allowing the model to capture dense temporal structures with minimal loss.
Recurrent Block (DDRB,
In general, the hidden state Hî and the cell state Cf in recurrent blocks at time L and scale are updated as:
F = Relu(eBN x 5(F) ((F(-11)))
1)),
(2)
C DDRB(FF,H1__1) C).1),
(3)
where BN means batch normalization preceding stage, and DDRB represents our proposed Decoupled Deformable-enhanced Recurrent Block, which encapsulates our novel integration of deformable spatial adaptation and temporal recurrence mechanisms.
F denotes the feature map obtained from the
3.4 Decoupled Deformable-enhanced Recurrent Layer
In event-based object detection tasks, event cameras exclusively detect moving objects, leading to suboptimal performance when relying solely isolated event frames in scenarios involving slow- moving or stationary targets. ConvLSTM [33), a neural network architecture specifically designed for temporal data sequences, addresses this limitation through its capacity for modeling longerange temporal dependencies. This characteristic enables enhanced detection of objects with low-motion or static states within dynamic scenes. Furthermore, the inherent integration of convolutional operations in ConvLSTM facilitates effective processing of spatial features within visual data. Particularly in event-driven object detection, where distinct spatial regions may contain heterogeneous motion patterns, ConvLSTMM assists the model in capturing these spatially distributed temporal characteristics, thereby substantially improving detection capabilities. In our proposed Deformable Enhanced Recurrent Layer, we also employ ConvI S for spatiotemporal modeling. However, diverging from conventional implementations, we identify that conventional ConvLSTM structures may underutilize extracted spatiotemporal features. To address this, we propose an innovative fusion module based on modulated deformable convolution [48) specifically designed for the characteristics of event data, by adopting wewiid-and-oonquue strategy. This architectural enhancement enables comprehensive exploitation of spatiotemporal information generated by ConvLSTM. particularly optimizing feature representation through adaptive receptive field adjustment and motion pattern alignment in complex scenarios. Meanwhile, since the recurrent layers are placed at low-dimensional stages, the low- dimensional features may contain task-irrelevant information in addition to the key information we need. The smoothing effect of the deformable convolution can effectively suppress such redundant information.
5

---

Specifically, the Decoupled Deformable-enhancee Recurrent Layer (DDRL) comprises a 2x downsampling module and Decoupled Deformable-enhannee Recurrent Block (DDRB) (as shown in Figure 3p. The downsampling module integrates convolutional layer with ker- nel size 5 and stride 2, followed by batch normalization (BN) and ReI.U activation, to perform preliminary spatial reduction. The DDRB includes recurrent module (Convl. STM), and a decoupled modulated deformable convolution specifically designed based on the characteristics of event data. ConvLSTM is employed to capture spatiotemporal infor- mation from consecutive event frames, while the decoupled modulated deformable convo- lution is used to fuse current frame features with historical spatiotemporal representations. Given the high temporal resolution event cam- eras, event data inherently contain fine-gres iie-igrined motion cues. Inspired by this, our design decou- ples per-pixel motion estimation from feature fusion. We leverage the intrinsic motion infor- mation in event data to learn the offsets Arg and modulated scalars Am, that 1) adapt the sam- pling grid of the modulated deformable convolu- tion to pixel-wise motion patterns in the scene, and 2) dynamically weight each sampling posi- tions based on their actual contribution to feature representation. The estimated offsets Arg and modulation masks Am; are applied to the fea- tures obtained from the preliminary fusion of the current frame and previous spatiotemporal repre- sentations, where the fusion is performed by con- catenating along the channel dimension followed by a 3x3 convolution for dimensionality reduc- tion. The deformable convolution subsequently adaptively samples features from the compressed representation using the learned Az spatially align with motion patterns, while Am, dynami- cally recalibrates sampling weights according motion relevance. This hierarchical fusion mech- anism effectively preserves fine-gr one-grained motion cues while eliminating feature redundancy, en- suring optimal utilization of event-driven spa- tiotemporal correlations.
As and
celorrable coneerien
Aeu or Op-
(a)
Squeere
Exilation
1xlst
1x1xc
(b)
Figure (a) Architecture of the Decoupled Deformable-enhanced Recurrent Layer. Ap- proaches for feature fusion typically employ direct combination through element -wise summation or channel-wise concatenation followed by convolu- tional integration, alternatively leverage atten- tion mechanisms for adaptive feature aggregation. Here, we leverage the divide-and-conge princi- ple to introduce modulated deformable convolu- tion into feature fusion, further exploiting tempo- ral information. (b) Architecture of the SE Block.
Given convolution kernel with K sampling locations, let wy and zy denote the weight and predefined offset of the & th location, respectively. In this work, we adopt 3 x 3 convolution kernel, i.c., K = and I& {(-1,-1) (-1, 0), (1. 1)). At time and scale s, the Decoupled Deformable-cnhame Recurrent Block (DDRB) can be represented as:
F = Relu BN (Conu x 3(Concat(FF) (F 1i-1))),
(4)
Ff" (z) = Σ wk F (a xx AzE) Amk.
(5)
Hi, C COW.LST
(6)
where Ary and Am, represent the learnable offset and modulation scalar for the k - th location,
respectively, both obtained from the current event feature F/ through two separate x 3 convolution layers. T denotes the spatiotemporal information from the previous time step t - 1 (H; or C2-1). In the experimental section, we investigate the performance differences when using the hidden state H; which represents the features of the previous time step, versus the memory cell C; I. which stores long-term spatiotemporal information. In addition, enable the model flexibly learn diverse deformation patterns across different spatial regions and enhance its representation capability for complex geometric transformations, we adopt grouped deformable convolutions. Specifically, input channels are divided into multiple groups (set to 8), with each group independently learning distinct deformation patterns. Furthermore, considering that the offset features generated by different groups may contribute unequally to the final task. we incorporate Squeeze-an and-ancitation (SE)

---

Block [14] after the grouped deformable convolution. The SE Block leverages global average pooling and fully connected layers to learn channel-wise attention weights, thereby enhancing informative channels and suppressing redundant ones, leading to optimized feature representations.
3.5 Scale-Specific Spatiotemporal Encoding
Multi-scale feature representation has been extensively employed in object detection frameworks to address scale variation challenges, particularly in detecting objects with diverse sizes. For instance, lower-scale branches preserve higher spatial resolution with shallower network depth, making them particularly suitable for detecting small or rapidly moving objects. As feature resolution progressively decreases through the network hierarchy, the expanded receptive fields enable context-aware detection of larger objects. This multi-scale temporal encoding structure allows the model to learn both fine- grained and coarse-grained motion features. In the present work, we process three low-dimensional multi-scale spatiotemporal features derived from preceding layers using three independent processing branches. Each branch implements scale-specific downsampling operations, followed by feature fusion through Feature Pyramid Network (24). This hierarchical architecture enables adaptive resolution adjustment across different network levels, effectively preserving critical information with enhanced flexibility while mitigating information degradation during downsampling.
Experiment
Dataset. The Genl automotive dataset (5) consists of 39 hours of event camera recordings with resolution of 304 x 240. It includes 228k annotated bounding boxes for vehicles and 28k for pedestrians, with available annotation frequencies of 1, 2, or Hz. Following the evaluation protocol of previous works [1 22 III. we discard bounding boxes with side lengths smaller than 10 pixels and diagonal lengths shorter than 30 pixels. Similarly, the Mpx dataset [31] focuses on driving scenarios but provides several months of higher-resolution (1280 x 720) daytime and nighttime recordings. It contains approximately 15 hours of event data, annotated at 30 or 60 Hz, with around 25 million bounding box labels distributed across three categories: vehicles, pedestrians, and two-wheelers. We adhere to the same evaluation protocol, removing bounding boxes with side lengths smaller than 20 pixels and diagonal lengths shorter than 60 pixels, and downsample the input resolution to 640 360. Unlike the Genl [5] and Mpx [31] datasets, the eTram dataset [37] is a traffic monitoring dataset collected from roadside perspective, thus exhibiting higher sparsity. e'Tram contains approximately 10 hours of data with a resolution of 1280 x 720, including around 2 million annotated bounding boxes across 8 categories, with annotations provided at 30 Hz. The preprocessing procedure of the e e'Tra dataset is similar to that of the Mpx dataset. For all datasets, mean Average Precision (mAP) (3 is considered as the primary metrie.
Implementation Details. During training, we adopt the ADAM optimizer [19 along with a OneCy- cle (36] learning rate scheduler, which linearly decays from its peak value. Following the strategy in RVT (T) we employ a mixed batch training technique, where standard Backpropagation Through Time (BPTT) is applied to half of the batch samples, while Truncated BPTT (TBPTT) is applied to the other half. Data augmentation includes random horizontal flipping, zoom-in, and zoom-out operations. The event representation is constructed ms time window. For the detection head, we utilize a Feature Pyramid Network (FPN) 24 for multi-scale feature fusion, along with the detection head from YOLOv6 (1], which incorporates distribution focal loss, classification loss, and regression loss.
5-channel voxel grid [45] based on 50
To compare against prior works on the Genl dataset, we train our models with a batch size of 6, sequence length of 21, learning rate of 2e - 4 for 400k itcrations. The training takes approximately days on single RTX 3090 GPU. On the 1 Mpx dataset, we train with a batch size of 8, sequence length of 5, and learning rate of 3e 4 for 800A iterations on a single RTX 3090 GPU, On the eTram dataset, the model is trained for 400k iterations with batch size of 8, sequence length of 5, and an initial learning rate of 3e- 4.
Benchmark Comparisons. We compare the proposed neural network architecture with previous works on the Genl 5 and Mpx 31 datasets, with the results summarized Table | From Table I. it can be concluded that the use of recurrent layers generally leads to better performance. achieves competitive results by replacing recurrent layers with state-space models 49 adaptively encodes spatiotemporal information from events, achieving
S5-VIT-B 50
(SSM) 12 35
ERGO-12
7

---

Table 1: Comparison with state-of-te the-hhhad methods on Genl and Mpx datasets.
Method
Params
Backbone
Gen1 mAP
Time (ms)
1Mpx mAP
Time (ms)
Asynet (26
11.4
Sparse CNN
14.5
AEGNN 32
20.0
GNN
16.3
Spiking DenseNet
4
8.2
SNN
18.9
Inception SSD 16
> 60"
CNN
30.1
19,4
34.0
45.2
RRC-Events 3
> 100"
CNN
30.7
21.5
34.3
46.4
MatrixLSTM 2
61.5
CNN RNN
31.0
YOLOv3 Events 18
> 60
CNN
31.2
22.3
31.6
49.4
RED 8
24.1
CNN RNN
40.0
16.7
43.0
39.3
ASTMNet
> 100
CNN RNN
35.6
48.3
72.3
ERGO-12
59.6
Transformer
50.4
69.9
46.0
100.0
RVT-B 11
18.5
Transformer RNN
47.2
10.2
47.4
11.9
Swin-T v2 25
21.1
Transformer RNN
45.5
26.6
45.5
34,8
Nested-T 43
22.2
Transformer RNN
46.3
20.6
46.0
33.5
GET-T 30
21.9
Transformer RNN
47.9
16.8
48.4
18.2
SAST-CB 29
18.9
Transformer RNN
48.2
22.7
48.7
23.6
S5-ViT-B 550
18.2
Transformer SSM
47.7
8.16
47.8
9.57
Ours
26.4
CNN RNN
52.7
8.80
49.1
13.3
Table 2: Comparison with state-of-the-art meth- ods on the traffic monitoring dataset e'Tram.
Table 3: Performance of different temporal in-
formation reuse methods on the Gen test set.
Fuse-Method
Feature-used mAP APso Para.
Method
Backbone
MAP
Time (ms)
(M)
RVT-B
Transforme+RNNN
29.5
10.88
Base
50.7
80.6
22.4
hidden
51.7
80.8
SAST-CB 29
Transforme++RNN
30,0
23.07
Directly fusion
cell
$2.0
81.5
26.6
S5-ViT-B 50
Transformer4SSM
29.3
14.84
hidden
52.2
81.5
DDConv
26.4
cell
52.7
81.5
Ours
CNN+RNN
33.0
13.05
hidden
52.3
81.7
DDConv
26.4
cell
52.7
81.7
state-of-the-ard performance without employing recurrent layers. It is worth noting that, unlike our CNN-RNN based approach, MatrixLSTM (2) applies LSTM units directly at the input level, while and ASTMNet 22 utilize recurrent layers only in deeper network stages.
RED 31
Our model achieves state-of-the-ard performance with mAP of 52.7 on the Gen1 dataset, and mAP of 49.1 on the Mpx dataset. Compared to other CNN-RNN methods in the table, our model achieves significantly higher performance while maintaining comparable or lower parameter counts. On the 1 Mpx dataset, it outperforms the second-bes, ASTMNet [22] by 0.8 mAP, and on the Genl dataset, it surpasses by 6.0 mAP. Notably, our model is trained from scratch without requiring any pre-trained weights.
To further validate the generalizability of our model across datasets, compared it on the sparser e'Tram dataset [37] against several Transformer- and SSM-based methods (RVT(IT), SAST (29). S5-VIT (50)) that have demonstrated strong performance the Genl (5) and Mpx (11 datasets. As shown in Table 2 our method attains the highest detection accuracy on the eTram dataset (37) while maintaining a high inference speed, achieving an mAP that 3.0 higher than that of the second-best SAST-CB 29
Ablation Study. (1) Effectiveness of Decoupled Deformable-enhanced Recurrent Layer. Table 3 systematically investigates the effectiveness of our proposed Decoupled Deformable-enhanee Recurrent Block (DDRB). The baseline model ('Base") employs conventional ConvLSTM layers (3 within our custom backbone architecture without temporal enhancement mechanisms, achieving 50.7 mAP on the Genl dataset. This baseline performance inherently validates the fundamental efficacy of our backbone design. Furthermore, attempt to enhance the base model by inserting

---

GT
Base
Directly fusion
Ours
Figure 4: Visualizations of the Decoupled Deformable-enhanced Recurrent Layer (DDRL) ablation
study on the Gen1 dataset.
x 5 convolution before each ConvLSTM to fuse the current event features with previous temporal information (called Directly Fusion in Table The results show that using the hidden state leads to a 1.0 mAP improvement, while using the cell state yields a 1.3 mAP increase, indicating that further exploiting prior temporal information can indeed enhance detection accuracy.
Inspired by the divide-and-od and-conquer principle, one of our principal innovations replaces these conven- tional convolutions with event event-based decoupled deformable convolutions (called DDConv in Table 3). This modification achieves superior performance (+00.5 mAP with hidden states, +00.7 mAP with cell states) while reducing parameter count, establishing an optimal balance between model efficiency and detection accuracy. In addition, we validate the effectiveness of integrating a lightweight SE block [14), which provides further performance improvement without increasing model complexity.
(2) Effectiveness the Proposed Backbone. To further investigate the effectiveness of the proposed backbone, we conducted experiments focusing on its key characteristics (Temporal Modeling at Lower Scales and Scale-Specific Spatiotemporal Encoding). All backbone variants this study utilized 5-channel event voxel 45 as input and shared YOLOv6 [21] detection head configuration. We maintained consistent feature downsampling ratios (8x, 16x, 32x) and corresponding channel dimensions (128, 256, 512) at the detection head input across all configurations. Except for the RVT 11 backbone, all implementations employed cell state for temporal feature enhancement. The experimental results are systematically presented in Table 4
Our initial investigation explored positioning the Decoupled (DDRB) in higher-dimensional spaces (as illustrated in Figure (a)). of channels at higher dimensions, the model parameters surged to 53.8M (an increase of 27.4M). However, this led to a 3.2 mAP drop, indicating that our low-dimensional spatiotemporal modeling strategy fundamentally aligns with event data characteristics. Additionally, we designed single- branch feature extraction backbone based layers unchanged, as shown Figure (b)). Although this slightly reduced the parameter count, also led to a 1.4 drop in mAP. Furthermore, we compared our backbone with the RVT [II] backbone one of the representative classical Transformer RNN methods. Without significantly increasing model complexity, designed backbone achieved improvements of 4.7 in mAp and 4.2 in APsu-
eeformable-enhancee Recurrent Block (a)). Due to the increased number
current backbone (with the placement of the recurrent
Visualizations. Figures and 5 present partial visualization results from the two ablation studies. The comparative analysis demonstrates that our methodology, which strategically leverages tempo- ral information through hivide-anddconques principle capitalizing on event data characteristics, enhances model robustness for object detection across varying motion velocities. Furthermore,

---

GT
Highlivvl Temporal Modeling
Sindle branch Encoding
RVT backoono
Ours
Figure 5: Visualization of the backbone ablation study the Genl dataset.
Dit made
Dolordde
Denagea
Table 4: Performance of different backbone types on the Genl test set.
(+)
Backbone Types
mAP APso Para.
(M)
Seorey
High-level Temporal Modeling
49.5
79.5
53.8
Single-branch Encoding
51.3
80.9
23.5
(b)
Figure 6: Backhone variants used in the ablation studies. (a) High-level Temporal Modeling. (b) Single-branch Encoding. The numbers after "downsample" indicate the downsampling ratio.
RVT 11 backbone
48.0
77.5
23.2
Ours
52.7
26.4
the incorporation of fine- fineigrained temporal propagation mechanisms enables better performance in challenging scenarios involving partial occlusions or small object detection.
Conclusion
This paper revisits the architectural design of event-based object detectors by emphasizing precise temporal modeling over increased complexity. We demonstrate that placing recurrent modules at lower spatial scales enables effective capture of dense temporal patterns present in raw event streams. To further enhance motion alignment and feature quality, we introduce the Decoupled Deformable- enhanced Recurrent Layer (DDRL), which decouples motion estimation from feature fusion and leverages deformable convolution adaptively align motion while suppressing task-irrelevant noise in liw-dimensional features. Combined with scale-specific downsampling and feature fusion, our models
CNN-RNN architecture achieves competitive or superior performance to transformer-bate without relying on global attention mechanisms. These results highlight that enhancing temporal modeling at the feature extraction stage is key to advancing event-based vision and point toward designing temporally-awee aware yet structurally simple backbones for sparse, asynchronous data.
10

---

Acknowledgment
This work partially supported by grants from the National Natural Science Foundation of China under contract No. 62302041.
References
Julian Büchel, Gregor Lenz, Yalun Hu, Sadique Sheik, and Martino Sorbaro. Adversarial attacks on spiking convolutional neural networks for event-based vision. Frontiers Neuroscience, 16:1068199 2022.
[2] Marco Cannici, Marco Ciccone, Andrea Romanoni, and Mattco Matteucci. Asynchronous convolutional networks for object detection in neuromorphic cameras. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition Workshops, pages 0-0, 2019.
[3] Nicholas F. Y. Chen. Pseudo-labels for supervised learning dynamic vision sensor data, applied to object detection under ego-motion. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2018.
[4] Loïc Cordone, Benoît Miramond, and Philippe Thierion. Object detection with spiking neural networks on automotive event data. In 2022 International Joint Conference on Neural Networks (IJCNN), pages 1 1-8. IEEE, 2022.
Pierre De Toumemirr, Davide Nitti, Etienne Perot, Davide Migliore, and Amos Sironi. A large scale event-based detection dataset for automotive. arXiv preprint arXiv:2001.08495 2020.
[6] Ziluo Ding, Rui Zhao, Jiyuan Zhang, Tianxiao Gao, Ruiqin Xiong, Zhaofei Yu, and Tiejun Huang. Spatio- temporal recurrent networks for event-based optical flow estimation. In Proceedings of the AAAI conference on artificial intelligence, volume 36, pages 525-533, 2022.
[7] Alexey Dosovitskiy. Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn. Xiaohua Zhai. Thomas Unterthiner. Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition scale. arXiv preprint arXiv:2010. 1929, 2020.
[8] Guillermo Gallego, Tobi Delbrück, Garrick Orchard, Chiara Bartolozzi, Brian Taba, Andrea Censi, Stefan Leutenegger, Andrew Davison, Jörg Conradt, Kostas Daniilidis, and Davide Scaramuzza. Event-based vision: A survey, IEEE Transactions on Puttern Analysis and Machine Intelligence, 44(1):154-1888 2022.
[9] Daniel Gehrig. Henri Rebecq. Guillermo Gallego, and Davide Scaramuzza. Asynchronous, photometric feature tracking using events and frames. In Proceedings of the European Conference on Computer Vision (ECCV), pages 750-765, 2018
[10] Daniel Gehrig and Davide Scaramuzza. Pushing limits asynchronous graph-based object detection with event cameras. arXiv preprint arXiv:2211.12322 2022.
[11] Mathias Gehrig and Davide Scaramuzza. Recurrent vision transformers for object detection with event cameras. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 13884-13893, 2023.
[12] Albert Gu, Karan Goel, and Christopher Ré. Efficiently modeling long sequences with structured state spaces. arXiv preprint 1..00921 2021.
[13] Sepp Hochreiter and Jürgen Schmidhuher. Long short-term memory. Neural Computation, 735:1735- 1997.
[14] Jie Hu, Li Shen, and Gang Sun. Squeeze-and-excitaaiin networks. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 7132-7141, 2018
[15] Yuhuang Hu, Tohi Delhruck, and Shih-Chii Liu. Learning exploit multiple vision modalities by using grafted networks. In European Conference on Computer Vision, pages 85-101. Springer, 2020.
[16] Massimiliano Iacono, Stefan Weber, Arren Glover, and Chiara Bartolozzi. Towards event-driven object detection with off-the-shelf deep learning. In 2018 JEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1-9, 2018
[17] Sergey loffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. JMLR.org, 2015. 2015.

---

[18] Zhuangyi Jiang, Pengfei Xia, Kai Huang, Walter Stechele, Guang Chen, Zhenshan Bing, and Alois Knoll. Mixed frame-Jevent-driver fast pedestrian detection. In 2019 International Conference on Robotics and Automation (ICRA), pages 8332-8338, 2019.
[19] Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. Computer Science, 2014.
[20] Xavier Lagorce. Garrick Orchard, Francesco Galluppi. Bertram Shi, and Ryad Benosman. Hots: a hierarchy of event-based time-surfaces for pattern recognition. IEEE trunsactions on pattern analysis and machine intelligence, 39(7): (77):134:: 10111346-1335 1355,999,,... 2016.
[21] Chuyi Li, Lulu Li, Hongliang Jiang. Kaiheng Weng, Yifei Geng, Liang Li, Zaidan Ke, Qingyuan Li, Meng Cheng, Weiqiang Nie, et al. Yolov6: A single-stage object detection framework for industrial applications. arXiv preprint arXiv:2209.021 02976, 2022.
[22] Jianing Li, Jia Li. Lin Zhu, Xijie Xiang, Tiejun Huang, and Yonghong Tian. Asynchronous spatio-temporal memory network for continuous event-based object detection, IEEE Trunsactions on Image Processing, 31:2975-2987, 2022.
[23] Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer vision-ECCC ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings, part 13. pages 740-755. Springer, 2014.
[24] Shu Liu, Lu Qi, Haifang Qin, Jianping Shi, and Jiaya Jia. Path aggregation network for instance segmenta- tion. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 8759-8768, 2018.
[25] Ze Liu, Han Hu, Yutong Lin, Zhuliang Yao. Zhenda Xie, Yixuan Wei, Jia Ning, Yue Cao. Zheng Zhang, Li Dong et al. Swin transformer v2: Scaling up capacity and resolution. In Proceedings of the IEFF/CVF conference on computer vision and pattern recognition, pages 12009-12019, 2022.
[26] Nico Messikommer, Danicl Gehrig, Antonio Loquercio, and Davide Scaramuzza. Event-basee asyn- chronous sparse convolutional networks. In European Conference on Computer Vision, pages 415-431. Springer, 2020.
[27] Anton Mitrokhin, Comelia Fermüller, Chethan Parameshwara, and Yiannis Aloimonos. Event-based moving object detection and tracking. In 2018 JEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1-9, 2018.
[28] Diederik Paul Moeys, Federico Corradi, Emmett Kerr, Philip Vance, Gautham Das, Daniel Neil, Dermot Kerr, and Tobi Delbrück. Steering predator robot using mixed framelevent-driven convolutional neural network. In 2016 Second International Conference Event-bused Control, Communication, and Signal Processing (EBCCSP), pages 1-8, 2016.
[29] Yansong Peng, Hebei Li, Yucyi Zhang, Xiaoyan Sun, and Feng Wu. Scene adaptive sparse transformer event- object detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 16794-16804, 2024.
[30] Yansong Peng, Yueyi Zhang, Zhiwei Xiong, Xiaoyan Sun, and Feng Wu. Get: Group event transformer for event-based vision. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6038 6048, 2023.
[31] Etienne Perot, Pierre De Tournemire, Davide Nitti, Jonathan Masci, and Amos Sironi. Learning detect objects with a 1 megapixel event camera. Advunces in Neural Information Processing Systems. 33:16639-1663 166522,91162,2211 2020.
[32] Simon Schaefer, Daniel Gehrig, and Davide Scaramuzza. Aegnn: Asynchromous event-based graph neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12371 12381, 2022.
[33] Xingjian Shi, Zhourong Chen, Hao Wang, Dit-Yan Yeung. Wai-Kin Wong, and Wang-chun Woo. Con- volutional Istm network: machine learning approach for precipitation nowcasting. Advances in neural information processing systems, 28, 2015.
[34] Amos Sironi, Manuele Brambilla, Nicolas Bourdis, Xavier Lagorce, and Ryad Benosman. Hats: Histograms of averaged time surfaces for robust event- based object classification. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 2018.
12

---

[35] Jimmy TH Smith, Andrew Warrington, and Scott W Linderman. Simplified state space layers for sequence modeling, arXiv preprint (XXiv:2208.04933 2022.
[36] Leslie N Smith and Nicholay Topin. Super-convergence: Very fast training of neural networks using large learning rates. In Artificial intelligence and machine learning for multi-domain operations applications. volume 11006, pages 369-386. SPIE, 2019.
[37] Aayush Atul Verma, Bharatesh Chakravarthi, Arpitsinh Vaghela, Hua Wei, and Yezhou Yang. etram: Event-baser based traffic monitoring dataset. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 22637 22646, 2024.
[38] Dongsheng Wang, Xu Jia, Yang Zhang, Xinyu Zhang, Yaoyuan Wang, Ziyang Zhang, Dong Wang. and Huchuan Lu. Dual memory aggregation network for event-based object detection with learnable representation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 2492-2500, 2023.
[39] Xiao Wang, Chao Wang. Shiao Wang, Xixi Wang. Zhicheng Zhao, Lin Zhu, and Bo Jiang. Mambaevt: Event stream based visual object tracking using state space model. IEEF Transactions Circuits and Systems for Video Technology, 2025.
[40] Nan Yang. Yang Wang. Zhanwen Liu, Meng Li, Yisheng An, and Xiangmo Zhao. Smamba: Sparse mamba for event-based object detection. arXiv preprint arXiv:2501.119) 2025.
[41] Friedemann Zenke and Surya Ganguli. Superspike: Supervised learning in multilayer spiking neural networks. Neural computation, 30(6):1514-1511 2018.
[42] Pengjie Zhang. Lin Zhu, Xiao Wang, Lizhi Wang. Wanxuan Lu, and Hua Huang. Ematch: A unified framework for event event-based optical flow and stereo matching. arXiv preprint arXiv:2407.21735, 2024.
[43] Zizhao Zhang, Han Zhang, Long Zhao, Ting Chen, Serean Ö Arik, and Tomas Pfister. Nested hicrarchical transformer: Towards accurate, data-efficient and interpretable visual understanding. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 3417 112-4425, 2022.
[44] Alex Zihao Zhu, Liangzhe Yuan, Kenneth Chaney, and Kostas Danilidiss. Ev-flownet: Self-supervised optical flow estimation for event-based cameras. arXiv preprint arXiv: (00,,880220999 2018.
[45] Alex Zihao Zhu Liangzhe Yuan, Kenneth Chaney, and Kostas Daniilidis. Unsupervised weentbbssed learning of optical flow, depth, and egomotion. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 989-997, 2019.
[46] Lin Zhu, Xianzhang Chen, Lizhi Wang, Xiao Wang, Yonghong Tian, and Hua Huang. Continuous time object segmentation using high temporal resolution event camera. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
[47] Lin Zhu, Xiao Wang, Yi Chang, Jianing Li, Tiejun Huang, and Yonghong Tian. Event-based vidco reconstruction via on computer vision and pattern recognition, pages 3594-3604, 2022.
spiking neural network. In Proceedings of the IEEE/CVF conference
[48] Xizhou Zhu, Han Hu, Stephen Lin, and Jifeng Dai. Deformable convnets v2: More deformable, better results. In Proceedings of the IEFF/CVF conference on computer vision and pattern recognition, pages 9308-9316, 2019.
[49] Nikola Zubić, Daniel Gehrig, Mathias Gehrig, and Davide Scaramuzza. From chaos comes order: Ordering event representations for object recognition and detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 12846-12856, 2023.
[50] Nikola Zubie, Mathias Gehrig, and Davide Scaramuzza. State space models for event cameras. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5819-5828, 2024.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Yes, the abstract and introduction have clearly stated the contributions of our work.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: Yes, the limitation is discussed in our appendix.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
14

---

Justification:
The paper does not include theoretical results
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
The code is included in the supplementary materials.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
15

---

Answer:
[Yes]
Justification:
The code is included in the supplementary materials.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.com cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https httpa/////iiee/eeeeebbiiiieeeee htt////ooeSumissseeeeee
7/nips.com wttp:ccpppbccccceee hww.bil/cguudeee.///eeee
swiiiiissipppcccccccooooooo for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification:
The training and test details are included in the experiment section.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification:
The experiment fixed dataset and fixed seeds, and does not involve
these.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
16

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Yes, all details have been provided.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: Yes, we have checked it.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
17

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Yes. Two datasets have been cited.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, papprswiiiiiie thcode com datastts has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
18

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: Yes, the training details of our model is presented.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/cooe/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
19

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy pattps://neuripge: /ttpaarrpp..cco/oofffeeeee/////s for what should or should not be described.
20

---

A
Supplementary Experiments
This section primarily provides an additional analysis of the ablation experiments related to the backbone, followed by an investigation of the placement of the Decoupled Deformable-enhanced Recurrent Layer (DDRL) through experimental evaluation.
A.1 Supplementary Ablation Experiment
During ablation studies the backbone architecture, devised two specialized variants, High- level Temporal Modeling and Single-branch Encoding, explicitly tailored to investigate its critical characteristics: Temporal Modeling at Lower Scales and Scale-Specific Spatiotemporal Encoding. We also compared our backbone with the backbone of the classic Transformer-RNNX method RVT ([]] However, we realized that high-dimensional temporal modeling allows for the parallel placement of the recurrent module (as shown in Figure 7 (a)). Therefore, under the same configuration, we conducted an additional experiment, High-level Temporal Modeling (Parallel), where Decoupled Deformablc-enhane Recurrent Block (obtained by removing the downsampling module from DDRL, as shown Figure (b)) is parallelly placed at high dimensions.
The experimental results are shown in Table 5 Whether the recurrent module is cascaded or parallelly placed at high dimensions, both configurations lead to increase in parameters accompanied by performance degradation. This may be because deep networks, through multiple downsampling and filtering operations, may overly smooth sparse event signals, weakening the recurrent module's 's ability to respond to critical temporal changes. Additionally, the effect of parallel placement of the recurrent module is less effective than cascading, possibly because in the parallel structure, each scale's recurrent module independently processes temporal information, lacking cross-scale temporal context transmission. The high-level semantic features are unable leverage the fine-grainn fine-grained temporal variations in the low-level high-resolution features, resulting in fragmented temporal modeling. In our approach, recurrent layers are continuously positioned at dower-dimensional stages, aiming to preserve fine-grained temporal information and cross-scale sequential context. Simultaneously. multi-branch feature extraction mechanism is introduced flexibly retain salient information while mitigating the loss incurred during the downsampling process.
Table Performance of different backbone types on the Gen1
5 test set.
Backbone-Types
mAP
AP50
Para. (M)
High-level Temporal Modeling
49,5
79.5
53.8
High-level Temporal Modeling (Parallel)
47.5
77.4
53.8
Single-branch Encoding
51.3
80.9
23.5
RVT I backbone
48.0
77.5
23.2
Ours
52.7
81.7
26.4
A.2 Analysis of the placement of the Decoupled Deformable-enhanced Recurrent Layer (DDRL)
To further explore the optimal placement of the Decoupled Deformable-enhanced Recurrent Layer, we attempted to place consecutively at different scales (as shown in Figure the experiment, we still use 5-channel event voxels (45 as input, with FPN 24 performing multi-scale feature fusion and the YOLOv6 (21] detection head conducting detection. Additionally, we ensure that the feature downsampling factors for each backbone input to the detection head are 8. 16, and 32, corresponding to channel numbers of 128, 256, and 5 512, respectively. The models were trained on the G [5] dataset for epochs with a batch size of and learning rate of 2e-4.
The experimental results are shown in Table 6 Low, Mid, and High represent the placement of the Decoupled eefformable-enhnnee Recurrent Layer at three different positions, from www-dimensional to high-dimensional. The numbers in parentheses indicate the downsampling factors corresponding to the features input to the three ConvL.TM 33 For example, when placed at the lowest dimension, the downsampling factors for the features input to ConvI are 2, 4, and 8, respectively. It can
be observed that placing the Decoupled Deformablc-enhanee Recurrent Layer at low dimensions
21

---

DDRB
DDRB
downsample (4)
downsample
downsample (2)
Nownssmppe (2)
read
(2)
DDRB
(a)
conv
offset
mask
hg and Ck
ConvENRELU
SE Block
ConVLSTM
deformable convolution
hx-1 CR-1
(b)
hR-1 and Ck-1
Figure 7: (a) High-level Temporal Modeling (Parallel). The number after 'downsample" indicates the downsampling ratio. (b) The architecture of Decoupled Deformable-enhancee Recurrent Block (DDRB).
yields better performance. This is likely because low- -ww-dimensional features typically preserve higher spatial resolution and finer-grained temporal information. Event camera data inherently sparse and consists of asynchronous event streams. Shallow recurrent modules can directly model short-term motion patterns (such as edge movement and local brightness changes) on low-level abstract features, avoiding temporal blur in deep features caused by multiple downsampling operations.
Table 6: Performance difference of DDRL. at different positions on the Gen
test set.
Placement
mAP
APso 50
Para. (M)
High (8, 16, 32)
49.5
79.5
53.8
Mid (4, 8, 16)
51.0
80.1
22.9
Low (2, 4, 8)
52.0
81.5
23.5
B Visualization
Visualization of temporal features. To more intuitively validate the effectiveness of our Decoupled enformable-enhance, Recurrent Layer for temporal feature enhancement, we visualize the features output by the first and second recurrent layers (as shown in the Figure and I0) and compare them with our base model (which uses ConvLSTN in the recurrent layers without enhancement), not choose to visualize the features from the third recurrent layer because, after a certain degree of downsampling, the features became too abstract. can be observed that, since both models contain recurrent layers, the model is able to retrieve information from past events, even when objects
did
22

---

Decoupled Deformable enhanced Recurrett Layer
Decoupled Deformable- enhanced Recurrent Layer
Decoupled Deformable- enhanced Recurrent Layer
downsample (4)
head
Decoupled Deformable- enhanced Recurrent Layer
enhanced Recurrent Laver
Decoupled Deformable- enhanced Recurrent
downsample (2)
Decoupled Deformable.
downsample (2)
head
Layer
(
Deformable- enhanced Recurrett Layer
enhanced Recurrnt
enhanced Recurrnt
Decoupled
Deformable-
Decoupled
Decoupled Deformable-
ample (2)
downsample (2)
head
Layer
Laver
Conv
(c)
Figure 8: (a), (b), and (c) represent the placement the Decoupled Deformable-enhancee Recurrent Layer from high-dimensional to oowddimensional, corresponding to High, Mid, and Low, respectively, The number after 'downsample' indicates the downsampling ratio. The 'Conv" used in (c) is specificallyy employed for channel dimension reduction.
gradually disappear in some scenarios. However, from the lower-dimensional features output by the first recurrent layer, is evident that after enhancing the temporal features with our method, the details of the features richer and the noise is significantly reduced. This may be the result of combining the motion information from events with deformable convolutions. From the features output by the second recurrent layer, it is apparent that, compared to the base model, the enhanced model is able to focus more on the regions where objects are present.
As mentioned in the previous work RED (31. one of the important reasons for placing recurrent layers at high scales is to prevent recurrent layers from dynamically modeling low-level features that unnecessary for the given task. However, the decoupled deformablc-cnhane module added before the recurrent layers can greatly reduce these unnecessary features through the smoothing effect of deformable convolution.
Visualization of deformable offsets. further validate that the proposed Decoupled Deformable- enhanced Recurrent Layer (DDRL) effectively learns motion-aware spatial alignment, we visualize learned deformable offsets (Arr.) from the first DDRL using heat maps. The visualization performed on the e'Tram dataset [37]. traffic monitoring benchmark where the event camera remains almost stationary during recording. This static setup allows us to more intuitively observe the motion patterns of moving vehicles and pedestrians without interference from ego-motion.
23

---

events
base level1
ours ours_levell level1
base level2
ours, ours_level2 level2
t-4
t-3
t-2
t-1
Figure 9: Visualization heatmap the features output by the first (levell) and second (leve12) recurrent layers on Genl [5). "base" is the variant of our model where temporal enhancement is not applied in the recurrent layers.
As illustrated in Figure 11 the learned offsets exhibit consistent and coherent motion directions aligned with object trajectories, confirming that the deformable module adaptively follows scene dynamics. Specifically, regions corresponding moving vehicles show large, structured offsets oriented along the motion paths, while static background areas exhibit minimal displacement. This behavior demonstrates that the DDRL effectively decouples per-pixel motion estimation from feature fusion, adaptively aligning spatiotemporal features across consecutive event frames. These results empirically support our design motivation: the deformable convolution in DDRL captures fine- grained motion cues and spatially aligns event features according to actual object motion, thereby enhancing temporal consistency and detection accuracy in dynamic scenes.
C Limitations
Although the model designed is capable of effectively leveraging the sparsity of events well as the temporal and motion information contained within the events, resulting in promising performance, it also introduces certain degree of computational complexity. Whether methods such as sparse convolutions can be employed to reduce the computational load without compromising performance question are currently considering.
Morcover, we employ a relatively simple event representation that, although containing some temporal information, does not fully exploit the potential of event- based data. In future work, we aim to explore an event representation incorporating richer temporal information. thereby reducing the dependency of event based object detection the parameter count and complexity of recurrent layers.
24

---

events
ours level1
base level2
base_level2
ours level2
t-4
t-3
t-2
t-
Figure 10: Visualization heatmap of the features output by the first (levell) and second (level2) recurrent layers on Genl [5]. 'base' is the variant of our model where temporal enhancement not applied in the recurrent layers.
sources
offsets
Figure 11: Visualization of the learned offsets from the first DDRL layer along with the corresponding source voxel visualization on eTTram 37
25


Reviews:
[{"review_id": "HQmKwZoTjL", "reviewer": "Reviewer_S6Df", "summary": "This paper considers object detection using event-camera data. The authors argue that temporal information needs to be captured at low spatial scales before the extraction of high-dimensional convolutional features reduces the spatial resolution. The key idea is to introduce event-recurrent (RNN) modules at low spatial scales first, followed by scale-specific (CNN) feature extraction and a feature pyramid network (FPN) for object detection. The proposed RNN-CNN architecture operates over a 3D spatio-temporal voxel grid of event polarities, applies decoupled deformable convolution to fuse the current frame features with historical features, a ConvLSTM layer to capture spatio-temporal features from consecutive event frames, and finally an FPN to fuse multi-scale features. The method is evaluated on two of the widely used datasets for event-based object detection: Gen1 and 1Mpx in comparison to several recent Transformer, Transformer+RNN, GNN, and CNN+RNN architectures.", "questions": "1. My most critical suggestion is to add a generalization experiment, e.g., train on one dataset and test on the other. For example, this paper https://arxiv.org/abs/2505.12908 does this in their Table 2. It would be nice to see the performance of the proposed method in a similar experiment.\n \n2. The paper is repetitive about the high-level points that temporal patters in event data appear in low-level features and are best captured before spatial abstraction. This point and several related ones are made multiple times in the Intro, Related Works, Method sections. On the other hand, the review of related work is quite brief and very few qualitative results are presented. I recommend that the authors reduce the repetitiveness of the high-level motivation for the approach and instead add more of: technical details, review of additional related works, qualitative results.\n\n3. The authors should include the time and accuracy results for all methods in Table 1. Also, the best and second-best method should be highlighted for the time results as well.\n \n4. Minor comments:\n  - The meaning of hidden state, cell state, and BN are not clear the first time they are introducing in eq. (2), (3). These elements are not explained clearly until Sec. 3.4\n  - The references have multiple problems that need to be addressed, e.g., [7] is missing a publication venue, [10] includes et al. in the author list (not preferable), [17], [18], etc. include the publication year and IEEE twice (repetition is unnecessary), etc.", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths:\n + The proposed architecture is simple, well motivated, and performs well. I find the proposed design using several recurrent layers with deformable convolution and LSTM for the temporal events simple and effective. The consideration of both a hidden state and a memory cell in the LSTM model is nice.\n \n + The performance of the method on the Gen1 and 1Mpx datasets is very good in terms of both accuracy and time in comparison to strong recent baselines.\n\n + The ablation study results are interesting and, importantly in my opinion, underscore that the proposed design is effective, even without deformable convolution or Squeeze-Excite blocks in the model, and in comparison to Transformer-RNN [13].\n \n \n## Weaknesses:\n - The use of deformable convolution in the recurrent layer design raises questions about generalization. It is important that the authors evaluate the performance of the model when trained on one dataset and tested on a different one. \n\n - The presentation in Sec. 1-3 is somewhat repetitive regarding the proposed design. Reducing the repetitiveness will allow more space for presenting additional technical details, adding more quantitative (generalization) and qualitative results, and improving the qualitative results.\n \n - The qualitative results in Fig. 5 and 6 are hard to see and offer limited value. What are we supposed to observe? How does the visualization show anything about the feature processing and integration by the different models? The evaluation can be strengthened by providing better/additional qualitative results.", "was_revised": true, "final_justification": "This is a well written paper that proposes a simple and effective architecture for processing event camera data using recurrent layers at low spatial scales, followed by CNN feature extraction and fusion of current and historical features from consecutive frames. The proposed architecture performs very well on the Gen1 and 1MPx datasets in terms of accuracy and speed and is supported by thorough ablation experiments. In the rebuttal, the authors offered to add results which show generalization of the deformable convolution in the current layer design from one training set to a different test set. This is an important supplement to the evaluation. Overall, I think this is a strong paper and the authors have responded well and in detail to the reviewers' questions.", "rebuttal": "**[W1, Q1 generalization experiment]:** The use of deformable convolution in the recurrent layer design raises questions about generalization. It is important that the authors evaluate the performance of the model when trained on one dataset and tested on a different one.\n\n**[A1]:**\n\nThank you for raising this important question regarding the generalization of deformable convolution in the recurrent layer design. We agree that cross-dataset evaluation is essential to verify the model's generalization ability. We conducted generalization experiments using our model, carrying out cross-dataset generalization between the Gen1 and 1Mpx datasets, and compared them with two advanced Transformer and Mamba models (RVT, SSM). We directly used the weights trained on one dataset for testing on the other dataset without fine-tuning the weights.\n\nIt can be observed that despite using deformable convolution to dynamically learn offsets in event data, the generalization ability of our model is still comparable to, or even better than, the other two types of methods, especially in the generalization experiment from Gen1 to 1Mpx. \n\n**Generalization: 1Mpx → Gen1**\n\n| method | mAP  | mAP50 |\n|--------|------|--------|\n| Ours   | 12.2 | 35.6   |\n| RVT-B  | 12.4 | 34.0   |\n| S5-B   | 11.8 | 31.5   |\n\n**Generalization: Gen1 → 1Mpx**\n\n| method | mAP  | mAP50 |\n|--------|------|--------|\n| Ours   | 28.1 | 57.4   |\n| RVT-B  | 25.9 | 50.6   |\n| S5-B   | 21.6 | 42.9   |\n\n---\n\n**[W2, Q2]:** The paper is repetitive about the high-level points that temporal patterns in event data appear in low-level features and are best captured before spatial abstraction. This point and several related ones are made multiple times in the Intro, Related Works, Method sections. On the other hand, the review of related work is quite brief and very few qualitative results are presented. I recommend that the authors reduce the repetitiveness of the high-level motivation for the approach and instead add more of: technical details, review of additional related works, qualitative results.\n\n**[A2]:** Thank you very much for your constructive suggestion. Based on your feedback, we will make the following improvements:\n\n- **Reducing redundancy:** We will revise and streamline the Introduction and Related Work sections to eliminate repetitive statements about our method. The discussion on the decoupled deformable-enhanced recurrent layer and low-level spatiotemporal modeling will be condensed and clarified within the Method section.\n\n- **Expanding related work:** The original Related Work section was kept concise due to space limitations. We will expand this section to include recent works such as ERGO-12 and S5-ViT, providing a more comprehensive overview of event data modeling approaches.\n\n- **Qualitative visualization analysis:** Figures 5 and 6 present qualitative results from our ablation study to demonstrate the effectiveness of our proposed module. We acknowledge that the detection boxes may have been too thin, making it difficult to observe differences. In several scenes, other ablated variants exhibit missed detections or false positives.\n\n---\n\n**[W3, Q3 Table1]:** The authors should include the time and accuracy results for all methods in Table 1. Also, the best and second-best method should be highlighted for the time results as well.\n\n**[A3]:** The missing entries in Table 1 correspond to earlier works published before the release of the 1Mpx dataset, and these methods generally exhibit lower performance. Additionally, their original papers did not report inference time, which makes it difficult for us to provide fair or reproducible runtime comparisons.\n\n---\n\n**[Q4]:** Minor comments  \n**[A4]:** Thank you for your careful reading. We have addressed your suggestions one by one and thoroughly checked the manuscript for any other potential issues."}, {"review_id": "plITbULLnw", "reviewer": "Reviewer_vrrf", "summary": "This paper proposes a hybrid CNN–RNN architecture for event-based object detection that emphasizes fine-grained temporal encoding at early stages. Raw event streams are discretized into 5 temporal bins over 50 ms, then processed by a shared stem and three Decoupled Deformable-enhanced Recurrent Layers (DDRLs). Each DDRL combines a ConvLSTM with a modulated deformable convolution to align per-pixel motion before spatial downsampling. Multi-scale spatiotemporal features from three resolution branches are fused via an FPN and passed to a YOLOv6-based detection head. Experiments on the Gen1 (304×240) and 1 Mpx (1280×720) driving datasets show state-of-the-art mAPs of 52.7% and 49.1%, respectively, with comparable parameter count and FLOPs.", "questions": "1. Have you conducted an ablation over voxelization granularity (e.g., bins ∈ {5, 8, 10}, window ∈ {20 ms, 50 ms, 100 ms}) to confirm that the chosen 5-bin, 50 ms discretization sufficiently captures the fastest event dynamics?\n2. Could you visualize the learned deformable offsets (∆xₖ), such as heat maps or flow fields, to validate that the deformable module aligns motion as intended?\n3. Please compare or discuss your approach relative to SODFormer (Li et al., 2023) and the Time-Surface Pyramid (Manderscheid et al., 2019). Benchmarking against these fine-grained temporal-encoding methods would help clarify the paper’s novelty and positioning.\n4. Can you report end-to-end inference latency, peak GPU memory usage (batch 1), and a small/medium/large object AP breakdown on both Gen1 and 1 Mpx inputs, measured on specified hardware?\n5. Would moving one or two key supplementary results (e.g., the Table S1/S2 ablations or a representative offset heat map) into the main manuscript improve transparency around your architectural choices?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The early-stage ConvLSTM effectively captures temporal cues before significant spatial pooling, preserving fine-grained motion information.\n2. Combining ConvLSTM with a modulated deformable convolution in each DDRL adaptively aligns pixel-wise motion, reducing blur in fast-moving regions.\n3. The divide-and-conquer architecture cleanly separates motion estimation from feature fusion, which suits the asynchronous nature of event data.\n4. The model delivers state-of-the-art mAP on both Gen1 and 1 Mpx driving benchmarks, demonstrating robustness across resolutions.\n5. Comprehensive ablation studies (e.g., DDRL placement, branch design) substantiate key architectural choices.\n\nWeaknesses:\n1. The fixed 5-bin, 50 ms voxelization may not capture the fastest or smallest event dynamics, yet no ablation evaluates finer temporal discretization.\n2. The related work omits key fine-grained temporal-encoding methods (e.g., SODFormer, Time-Surface Pyramid), leaving the paper’s positioning in this space unclear.\n3. There is no visualization or quantitative analysis of the learned offsets (∆xₖ), nor any comparison to optical-flow ground truth to validate that motion alignment is actually occurring.\n4. Combining deformable convolution with ConvLSTM could over-smooth spatial detail, but no effective-receptive-field or smoothing analysis is provided.\n5. Independently processed branches are fused via a standard FPN without any temporal-state synchronization mechanism, and no small/medium/large AP breakdown or branch-ablation confirms the benefits for small or rapid objects.\n6. The efficiency evaluation is incomplete: “Time (ms)” in Table 1 is ambiguous, and there are no hardware-specific end-to-end latency, peak memory, or throughput measurements to support the “compact yet accurate” claim.", "was_revised": true, "final_justification": "The rebuttal and additional experiments have fully addressed my earlier concerns, particularly regarding voxelization ablation, method positioning, motion offset visualization, smoothing analysis, and efficiency breakdowns. These additions strengthen the empirical evidence and clarify the paper’s contributions. In light of the clarifications and new results, I am increasing my rating to 5 (Accept).", "rebuttal": "**[W1, Q1]:** The fixed 5-bin, 50 ms voxelization may not capture the fastest or smallest event dynamics, yet no ablation evaluates finer temporal discretization.\n\n**[A1]:** Thank you for your valuable comments. We have added an ablation study on the number of voxel bins and the temporal window size. To reduce data generation and training time, we randomly selected one-third of the Gen1 training set for training and evaluated on the full Gen1 test set. All other settings were kept consistent with those in the main paper. Our experiments are divided into two groups:\n\n- **Fixed number of bins (5), varying temporal window sizes (20ms, 50ms, 100ms):**  \n  The results show that a 50ms window yields relatively better performance. This may explain why several prior works such as RED, RVT, and SSM have also adopted 50ms as the default window size.\n\n- **Fixed temporal window size (50ms), varying number of bins (5, 8, 10):**  \n  The results indicate that increasing the number of bins does not lead to significant performance gains. This may be due to the sparse nature of event data—when using voxel representation, 5 channels are already sufficient to capture most of the meaningful information available in the event stream.\n\n| bins | window (ms) | mAP  | mAP50 |\n|------|-------------|------|--------|\n| 5    | 50          | 45.4 | 74.3   |\n| 5    | 20          | 44.9 | 74.7   |\n| 5    | 100         | 43.6 | 72.0   |\n| 8    | 50          | 45.0 | 74.0   |\n| 10   | 50          | 45.4 | 75.2   |\n\n---\n\n**[W2, Q3]:** Please compare or discuss your approach relative to SODFormer (Li et al., 2023) and the Time-Surface Pyramid (Manderscheid et al., 2019). Benchmarking against these fine-grained temporal-encoding methods would help clarify the paper’s novelty and positioning.\n\n**[A2]:** Thank you for your suggestion. Based on your feedback, we have improved the Related Work section. Compared to SODFormer, both methods aim to model temporal dynamics in event data, but differ in design:\nSODFormer performs event - frame fusion using asynchronous attention to compensate for frame degradation under fast motion or low light. Our method focuses on event - only modeling, emphasizing early - stage temporal encoding to capture fine - grained motion without relying on frame input. SODFormer models long - term dependencies in deep layers via global attention, which is computationally heavy. In contrast, we apply recurrent modeling at lower feature stages, better preserving temporal cues like fast edge motions that tend to degrade after downsampling. Regarding Time - Surface Pyramid, it targets corner detection by building velocity - invariant patterns through local decay, aiming to normalize static temporal changes. Our DDRL module, by combining deformable convolution and recurrence, directly models motion dynamics and better serves event - based object detection tasks\n\n---\n\n**[W3, Q2]:** There is no visualization or quantitative analysis of the learned offsets ($\\Delta$x ), nor any comparison to optical - flow ground truth to validate that motion alignment is actually occurring.\n\n**[A3]:** Thank you very much for your suggestion. We fully agree that visualizing the learned offsets of the deformable convolution is important for better understanding its behavior. We have visualized the offsets from the first deformable - enhanced block using heatmaps. We chose this layer because it retains a higher spatial resolution and the features are less abstract, making the offsets more interpretable. The visualization reveals that the offsets primarily capture motion information within the current frame. And we will include these offset visualizations.\n\n---\n\n**[W4]:** Combining deformable convolution with ConvLSTM could over-smooth spatial detail, but no effective-receptive-field or smoothing analysis is provided.  \n\n**[A4]:** You make a valid point — combining deformable convolution with ConvLSTM may indeed risk over-smoothing spatial details. To investigate this, we provided a visual comparison in the supplementary material (Figures S3 and S4) between features output with and without deformable convolution in the recurrent layers. The results show that while adding deformable convolution introduces some smoothing, it primarily suppresses irrelevant or noisy information that does not contribute to object detection. We will further enhance this analysis by adding more visualizations.  \n\n---\n\n**[W5, W6, Q4]:** Can you report end-to-end inference latency, peak GPU memory usage (batch 1), and a small/medium/large object AP breakdown on both Gen1 and 1 Mpx inputs, measured on specified hardware?  \n\n**[A5]:** Thank you for your suggestion. We re-conducted the tests on an Nvidia RTX 3090, comparing with advanced methods such as RVT and SSM. The batch size was set to 1, and the results on the Gen1 dataset are as follows:  \n\n| method | mAP(%) | AP_S(%) | AP_M(%) | AP_L(%) | time(ms) | peak memory(MB) |  \n| ------ | ------ | ------- | ------- | ------- | -------- | --------------- |  \n| RVT-B  | 43.1   | 34.1    | 50.4    | 43.5    | 8.07     | 435.14          |  \n| RVT-S  | 42.6   | 33.3    | 50.3    | 46.2    | 8.34     | 358.82          |  \n| RVT-T  | 40.2   | 30.8    | 47.8    | 44.6    | 7.18     | 309.38          |  \n| S5-B   | 45.7   | 36.5    | 53.1    | 51.1    | 9.19     | 1306.02         |  \n| S5-S   | 44.9   | 35.3    | 52.8    | 45.4    | 9.36     | 948.69          |  \n| ERGO   | 50.6   | 42.0    | 58.0    | 52.8    | 205      | 1351.61         |  \n| Ours   | 52.7   | 45.0    | 59.4    | 51.8    | 8.08     | 434.58          |  \n\nHere, AP_S, AP_M, and AP_L represent the detection accuracies for small, medium, and large objects, respectively. It can be seen that our method achieves the best performance while ensuring high inference speed and low GPU peak memory, especially in small object detection.  \n\n---\n\n**[Q5]:** Would moving one or two key supplementary results (e.g., the Table S1/S2 ablations or a representative offset heat map) into the main manuscript improve transparency around your architectural choices?  \n\n**[A6]:** Thank you very much for your valuable suggestion. We fully agree with your idea and will make some adjustments."}, {"review_id": "eX4dEunMmI", "reviewer": "Reviewer_u4BD", "summary": "This paper addresses the challenge of effectively modeling temporal dynamics for event-based object detection. It proposes a CNN-RNN hybrid architecture that enhances temporal modeling by placing recurrent modules at lower spatial scales, which allows for the capture of fine-grained temporal features in sparse event streams. In addition, the authors introduce a Decoupled Deformable-enhanced Recurrent Layer to decouple per-pixel motion estimation and feature fusion, along with a multi-scale approach for spatio-temporal encoding. The experimental results demonstrate that this method outperforms recent transformer-based models.", "questions": "See Weakness", "limitations": "No, it is recommended that the authors explore the limitations of the paper in detail", "rating": 2, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1. A new hybrid architecture, CNN-RNN, is proposed, offering a new perspective on event-based object detection by prioritizing early-stage temporal modeling.\n\n2. A scale-specific spatio-temporal encoding approach allows for more flexible and comprehensive feature extraction.\n\nWeaknesses:\n1. The proposed methods do not present new insights or theoretical advancements in the context of event-based representation learning, particularly in areas like asynchronous data processing or novel feature extraction techniques. This approach primarily improves on existing RNN-based models without introducing a groundbreaking shift in understanding the underlying event-based object detection problem.\n2. (1) Lack of recent state-of-the-art event-based object detection methods, such as: SpikeYOLO [1], EGSST [2], EventPillars [3], HsVT [4], SNN-ANN [5], etc；(2) Compared to recent Sstate-of-the-art methods, the performance improvement is limited; (3) Has the proposed method been tested for other alternative event representations (such as time surfaces, histogram)? (4) Can the proposed method be tested on the EvDET200K [6] dataset?\n\n[1] Integer-valued training and spike-driven inference spiking neural network for high-performance and energy-efficient object detection (ECCV2024)\n\n[2] EGSST: Event-based Graph Spatiotemporal Sensitive Transformer for Object Detection (NeurIPS 2024) \n\n[3] EventPillars: Pillar-based Efficient Representations for Event Data (AAAI2025)\n\n[4] Hybrid Spiking Vision Transformer for Object Detection with Event Cameras (ICML2025)\n\n[5] Efficient Event-Based Object Detection: A Hybrid Neural Network with Spatial and Temporal Attention (CVPR2025)\n\n[6] Object detection using event camera: A moe heat conduction based detector and a new benchmark dataset (CVPR2025)", "was_revised": true, "final_justification": "After carefully considering the authors' rebuttal, I still do not find the novelty of the work convincing. Although the authors have introduced some modifications, these appear to be more incremental refinements rather than substantive, novel contributions. Consequently, I maintain my original score.", "rebuttal": "**[W1]:** The proposed methods do not present new insights or theoretical advancements in the context of event-based representation learning, particularly in areas like asynchronous data processing or novel feature extraction techniques. This approach primarily improves on existing RNN-based models without introducing a groundbreaking shift in understanding the underlying event-based object detection problem.  \n\n**[A1]:** Thank you for your perspective. Our method mainly focuses on spatiotemporal feature extraction approaches from a new direction. Firstly, starting from the design of recurrent layers, we developed a decoupled deformable-enhanced module based on the characteristic that events contain motion information, aiming to reuse the state features in the recurrent module. This module integrates adaptive spatial sampling and temporal modeling capabilities, significantly improving the alignment effect of temporal features. Secondly, considering that low-scale features may contain richer motion details, we introduced recurrent layers at low scales and re-examined the temporal modeling method under scale-specific conditions. As mentioned in the previous work RED, one of the important reasons for placing recurrent layers at high scales is to prevent recurrent layers from dynamically modeling low-level features that are unnecessary for the given task. However, the decoupled deformable-enhanced module we added before the recurrent layers can greatly reduce these unnecessary features through the smoothing effect of deformable convolution (as shown in Supplementary Materials Figures S3 and S4).  \n\n- - -\n\n**[W2]:** (1) Lack of recent state-of-the-art event-based object detection methods, such as: SpikeYOLO [1], EGSST [2], EventPillars [3], HsvT [4], SNN-ANN [5], etc; (2) Compared to recent state-of-the-art methods, the performance improvement is limited; (3) Has the proposed method been tested for other alternative event representations (such as time surfaces, histogram)? (4) Can the proposed method be tested on the EvDET200K [6] dataset?  \n\n**[A2]:**  \n\n| method    | Gen1 mAP(%) | 1Mpx mAP(%) |  \n| --------- | ----------- | ----------- |  \n| SpikeYOLO | 40.4        | -           |  \n| EGSST-E   | 49.6        | 50.2        |  \n| EGSST-E-Y | 47.8        | 48.3        |  \n| HsvT-B    | 47.8        | -           |  \n| SNN-ANN   | 35          | 27          |  \n| Ours      | 52.7        | 49.1        |  \n\nThank you for mentioning many recent and meaningful SOTA methods. Compared with these SOTA methods you referred to, our method still remains competitive, especially in comparison with SpikeYOLO, HsvT-B, and SNN-ANN. EGSST-E has shown good performance on both the Gen1 and 1Mpx datasets. We believe this is partly attributed to its proposed dynamic label enhancement strategy, which improves the accuracy and adaptability of annotations by dynamically adjusting the time window for label matching. Additionally, it is worth noting that when the YOLO detection head is used to replace the RT-DETR detection head (i.e., EGSST-E-Y), the performance of EGSST decreases. Regarding (3) and (4), we are currently training on the EvDET200K dataset using histogram representation, and will share the results immediately during the discussion once available."}, {"review_id": "58NfK5D6Wi", "reviewer": "Reviewer_U1DD", "summary": "This manuscript proposes an CNN-RNN hybrid architecture tailored specifically for event-based object detection, highlighting the importance of early-stage, scale-aware temporal modeling. \n\nThe authors introduce recurrent modules at lower spatial scales to capture fine-grained temporal patterns inherent to sparse event streams. Central to their approach is the Decoupled Deformable-Enhanced Recurrent Layer (DDRL), a module that separates per-pixel motion estimation from feature fusion. \n\nThe architecture incorporates independent temporal downsampling branches at multiple resolutions, enabling flexible multi-scale spatiotemporal representation learning. \n\nExperiments conducted on two standard event-based object detection benchmarks, Gen1 and 1Mpx, demonstrate good performance over existing Transformer- and SNN-based methods, particularly in scenarios involving fast-moving or small-scale objects.", "questions": "1. Based on Weakness #1: While your integration of recurrent modules at earlier stages is effective, the key building blocks (ConvLSTM, deformable convolution, and FPN) are well-established. Can you clearly elaborate on which aspects of these modules were specifically adapted or uniquely designed for event-based detection in your work?\n\n---\n\n2. Based on Weakness #2: You evaluated your method on two standard datasets (Gen1 and 1Mpx). How well do you expect your architecture’s advantages to generalize beyond these specific benchmarks? Have you considered or tested scenarios with significantly different event distributions or sensor characteristics? The recent trend has shifted to incorporating more RGB cues in event-based detection frameworks. Can you discuss and compare your method (perhaps in event-frame fusion mode) with existing methods, such as DAGr [R1], RENet [R2], FlexEvent [R3], and/or CAFR [R4]?\n\n---\n\n3. Based on Weakness #3: Given the inherent complexity of recurrent and deformable convolution modules, have you performed a detailed runtime or complexity analysis under varying conditions (e.g., higher resolution or event density)? How might these computational considerations affect practical deployments on resource-constrained platforms?\n\n---\n\n4. Based on Weakness #4: Can you provide a more thorough discussion of scenarios or cases where your proposed DDRL module may perform poorly or have limited benefits? Understanding these boundary conditions would significantly strengthen the clarity and practical applicability of your contributions.\n\n---\nReferences:\n- [R1] Daniel Gehrig and Davide Scaramuzza. Low-latency automotive vision with event cameras. Nature, 629(8014):1034–1040, 2024.\n\n- [R2] Zhuyun Zhou, Zongwei Wu, Rémi Boutteau, Fan Yang, Cédric Demonceaux, and Dominique Ginhac. RGB-event fusion for moving object detection in autonomous driving. In IEEE International Conference on Robotics and Automation, pages 7808–7815, 2023.\n\n- [R3] Dongyue Lu, Lingdong Kong, Gim Hee Lee, Camille Simon Chane, and Wei Tsang Ooi. FlexEvent: Towards flexible event-frame object detection at varying operational frequencies. arXiv preprint arXiv:2412.06708, 2024.\n\n- [R4] Hu Cao, Zehua Zhang, Yan Xia, Xinyi Li, Jiahao Xia, Guang Chen, and Alois Knoll. Embracing events and frames with hierarchical feature refinement network for object detection. In European Conference on Computer Vision, pages 161–177, 2024.", "limitations": "- Although the method aims to address fine-grained temporal information effectively, scenarios with extremely sparse events—such as low-motion or stationary objects—may still challenge performance. The manuscript could benefit from further discussion on this point.\n\n- Despite demonstrating competitive inference speeds, the practical deployment of deformable and recurrent modules in real-time and resource-limited environments could still pose challenges. Acknowledging potential trade-offs between accuracy and latency/compute would enhance transparency regarding deployment feasibility.\n\n- Given the experiments’ constraints in scale (resolution downsampling and sequence length), performance scalability on much larger datasets or continuous event streams has not been explicitly verified. How the proposed approach scales up in terms of training time, computational resources, and memory consumption could be a limiting factor for widespread adoption.", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "### Strengths\n(+) The manuscript identifies a limitation in existing temporal modeling strategies for event-based object detection and convincingly motivates the need for early-stage temporal feature preservation. The introduction of recurrent modules at lower spatial scales is with a good motivation, aiming at addressing a previously under-explored aspect in event-based detection.\n\n(+) The proposed method achieves state-of-the-art results on the Gen1 and 1Mpx benchmarks. The performance improvements over recent Transformer-based event-only object detection methods underscore the efficacy of the proposed temporal modeling framework.\n\n(+) The manuscript is overall clearly structured, well-written, and technically sound.\n\n---\n\n### Weaknesses\n(-) Moderate novelty of individual components in the proposed framework. While the integration of recurrent modules at earlier stages is innovative for event-based detection, the individual components (ConvLSTM, deformable convolution, FPN-based feature fusion) have been well-established in other contexts. Thus, the technical novelty lies more in the thoughtful combination and adaptation rather than entirely new component inventions.\n\n(-) The evaluation is limited to two datasets, albeit popular and standard in event-based detection. Given the broad claims about scale-awareness and robust temporal modeling, experiments or at least discussions about generalizability to other event-based or event-frame fusion datasets could enhance the impact.\n\n(-) Although experiments report competitive inference speed, deformable convolutions and recurrent layers inherently add computational complexity. A deeper exploration of runtime implications, scalability to higher resolution event cameras, or computational trade-offs in real-world deployment scenarios would further strengthen the manuscript.\n\n(-) A more thorough discussion of the proposed method's limitations, failure modes, or scenarios where performance gains are minimal would provide valuable insights for future research.", "was_revised": true, "final_justification": "I have read the response, as well as the responses for review comments from other reviewers. Part of the concerns have been addressed. However, the evidence regarding the following concerns remains unsolved or only partially solved:\n- The design of combining blocks (ConvLSTM, deformable convolution, FPN) seems modular and incremental.\n- There is no proof of the proposed method across different benchmarks, distributions/sensor characteristics, or modalities.\n\nTherefore, I am leaning towards maintaining the rating given at the pre-rebuttal stage.\n\nBesides, I suggest that the authors should incorporate all clarifications, modifications, and new experiments into the revised manuscript to ensure that the quality meets what NeurIPS always looks for.", "rebuttal": "**[W1, Q1]:** Based on Weakness #1: While your integration of recurrent modules at earlier stages is effective, the key building blocks (ConvLSTM, deformable convolution, and FPN) are well-established. Can you clearly elaborate on which aspects of these modules were specifically adapted or uniquely designed for event-based detection in your work?  \n\n**[A1]:** Thank you for pointing out this important issue. We fully agree that ConvLSTM, deformable convolution, and FPN, as fundamental modules, have been widely used in other tasks. The innovation of our work indeed does not lie in inventing entirely new components, but in the targeted adaptation and reconstruction of these modules to meet the specific needs of event-based vision. The following are the core design adjustments we have made for asynchronous event data:  \n\n(1) **Decoupled deformable-enhanced recurrent module**: Considering the motion information in events, we designed a decoupled structure where deformable convolution first performs spatial sampling and alignment based on local motion, followed by accumulation in the temporal dimension via ConvLSTM. This sequence is specifically tailored to the sparsity of event data and its characteristic of containing motion information. The effectiveness of this design has been verified in ablation experiments.  \n\n(2) **Shallow recursion to preserve fine-grained temporal information**: Given the sparsity of events, shallow features may contain more critical information. Thus, unlike traditional methods that insert recurrent layers into deep semantics, we place the ConvLSTM modules earlier to better capture typical short-term motion patterns and local dynamics in event streams. Additionally, as mentioned in RED, a key reason for placing recurrent layers at high scales is to avoid recurrent layers from dynamically modeling low-level features unnecessary for the given task. However, the decoupled deformable-enhanced module we added before the recurrent layers can significantly reduce these unnecessary features through the smoothing effect of deformable convolution (as shown in Supplementary Materials Figures S3 and S4).  \n\n(3) **Time-aware multi-branch encoding and FPN fusion**: Our multi-branch backbone network performs temporal downsampling independently at each scale to avoid cross-scale interference. Features are fused only after scale-specific temporal encoding. Ablation experiments have also confirmed the effectiveness of the multi-branch design.  \n\n- - - \n\n**[W2, Q2]:** You evaluated your method on two standard datasets (Gen1 and 1Mpx). How well do you expect your architecture’s advantages to generalize beyond these specific benchmarks? Have you considered or tested scenarios with significantly different event distributions or sensor characteristics? The recent trend has shifted to incorporating more RGB data in event-based detection frameworks. Can you discuss and compare your method (perhaps in event-frame fusion mode) with existing methods, such as DAGR [R1], RENet [R2], FlexEvent [R3], and/or CAFR [R4]?  \n\n**[A2]:** Thank you for your suggestion. We are attempting to train on other datasets (such as EvDET200K) to demonstrate the generalization ability of our model, but this may take some time. Once we have results, we will report them immediately during the discussion. Additionally, thank you for pointing out fusion methods like DAGR, RENet, FlexEvent, and/or CAFR. These works demonstrate the great potential of fusing RGB with events, and we believe that combining work with multimodal frameworks is a direction well worth exploring. Although this paper currently focuses on pure event data input, the Decoupled Deformable-enhanced Recurrent Layer (DDRL) has good modular characteristics and can be embedded into the event branch to provide dynamic auxiliary information for RGB. Especially in scenarios where RGB is degraded, such as low light and blurriness, our method can provide temporal supplementary signals. We plan to focus on exploring this fusion direction in future work.  \n\n- - - \n\n**[W3, Q3]:** Given the inherent complexity of recurrent and deformable convolution modules, have you performed a detailed runtime or complexity analysis under varying conditions (e.g., higher resolution or event density)? How might these computational considerations affect practical deployments on resource-constrained platforms?  \n\n**[A3]:**  \n\n| resolution  | time(ms) | peak memory(MB) |  \n| ----------- | -------- | --------------- |  \n| 256*320     | 8.08     | 434.58          |  \n| 128*160     | 6.10     | 346.49          |  \n| 384*480     | 11.44    | 703.08          |  \n| 512*640     | 13.08    | 1109.56         |  \n| 768*960     | 28.18    | 2272.15         |  \n| 1024*1280   | 49.52    | 3888.79         |  \n\nThank you for this insightful question. We fully agree that for practical deployment, it is crucial to gain a deep understanding of the computational complexity and resource consumption impacts brought by recurrent and deformable modules, especially on resource-constrained platforms.  \n\nWe tested the inference time and memory usage under different input resolutions on a single RTX 3090, all based on the same model weights (trained on Gen1 with a resolution of 256×320). It can be seen that when the resolution is less than 512×640, our method still maintains relatively advanced inference time. Although the computational cost increases significantly at high resolutions (e.g., 1024×1280), considering the typical application scenarios of event cameras (such as real-time perception in autonomous driving, which usually adopts medium resolutions), our method can balance accuracy and efficiency under mainstream input sizes.  \n\n- - -\n\n**[W4, Q4]:** Can you provide a more thorough discussion of scenarios or cases where your proposed DDRL module may perform poorly or have limited benefits? Understanding these boundary conditions would significantly strengthen the clarity and practical applicability of your contributions.  \n\n**[A4]:** Thank you for your constructive question. We have found that the benefits of DDRL are limited in the following scenarios:  \n\n- **Highly crowded scenes with fast-moving and overlapping objects**: In scenes where multiple fast-moving objects are severely overlapping in both space and time (such as a congested intersection), deformable offsets may cause motion ambiguity when extracting motion information, leading to reduced alignment effectiveness and even feature blurring.  \n\n- **Static or near-static scenes**: Although the introduction of recurrent layers can to some extent improve the detection performance of slowly moving objects, due to the limited sequence length, the contribution of temporal modeling remains limited in certain scenarios."}]

Author rebuttal:

Meta-review: This paper proposes a hybrid CNN–RNN architecture for event-based object detection for modeling fine-grained temporal features. The main technical contribution is the Decoupled Deformable-Enhanced Recurrent Layer (DDRL), a module that separates per-pixel motion estimation from feature fusion. This is implemented using event-recurrent (RNN) modules at low spatial scales first, followed by scale-specific (CNN) feature extraction and a feature pyramid network (FPN) for object detection.

Most of the reviewers commended the strong empirical results, clear writing and extensive ablations. The reviewers did raise some concerns about the novelty, and comparison with SOTA, and all the reviewers have suggested that these should be included in the final version. The authors are encouraged to use this feedback to further improve the paper. 
Given the overall positive feedback from the reviewers, we propose an Accept for this paper.
Decision: Accept (poster)