Maximizing Asynchronicity in Event-based Neural Networks
Authors: Haiqing Hao
Nikola Zubic
Weihua He
Zhipeng Sui
Davide Scaramuzza
Wenhui Wang
Venue: ICLR 2026 Poster (2026)
Keywords: event camera
self-supervised learning
linear attention
linear RNN
neural network architectures
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
TL;DR: A new A2S framework inspired by NLP for learning and generating expressive and generalizable event features asynchronously.

Abstract: Event cameras deliver visual data with high temporal resolution, low latency, and minimal redundancy, yet their asynchronous, sparse sequential nature challenges standard tensor-based machine learning (ML). While the recent  asynchronous-to-synchronous (A2S) paradigm aims to bridge this gap by asynchronously encoding events into learned features for ML pipelines, existing A2S approaches often sacrifice expressivity and generalizability compared to dense, synchronous methods. This paper introduces EVA (EVent Asynchronous feature learning), a novel A2S framework to generate highly expressive and generalizable event-by-event features. Inspired by the analogy between events and language, EVA uniquely adapts advances from language modeling in linear attention and self-supervised learning for its construction. In demonstration, EVA outperforms prior A2S methods on recognition tasks (DVS128-Gesture and N-Cars), and represents the first A2S framework to successfully master demanding detection tasks, achieving a 0.477 mAP on the Gen1 dataset. These results underscore EVA's potential for advancing real-time event-based vision applications.

Published as a conference paper at ICLR 2026
MAXIMIZING ASYNCHRONICITY IN EVENT-BASED NEURAL NETWORKS
Haiqing Nikola Zubić?, Weihua He', Zhipeng Sui', Davide Scaramuzza', Wenhui Wang". State Key Laboratory of Precision Measurement Technology and Instrument, Department of Precision Instrument, Tsinghua University "Robotics and Perception Group, University of Zurich *Corresponding author wwh@tsinghua. edu. cn
Nikola
ABSTRACT
Event cameras deliver visual data with high temporal resolution, low latency, and minimal redundancy, yet their asynchronous, sparse sequential nature challenges standard tensor-based machine learning (ML). While the recent asynchronous-to synchronous (A2S) paradigm aims bridge this gap by asynchronously encoding events into learned features for ML pipelines, existing A2S approaches often sac- rifice expressivity and generalizability compared to dense, synchronous methods. This paper introduces EVA (EVent Asynchronous feature learning), novel A2S framework to generate highly expressive and generalizable event-by-event fea- tures. Inspired by the analogy between events and language, EVA uniquely adapts advances from language modeling in linear attention and self-supervised learn- ing for its construction. In demonstration, EVA outperforms prior A2S methods on recognition tasks framework successfully master demanding detection tasks, achieving 0.477 mAP on the Genl dataset. These results underscore EVA's potential for advanc- ing real-time event-based vision applications.
and N-Cars), and represents the first A2S
INTRODUCTION
Event cameras have advantages in high temporal resolution (up to 1 uS) and low spatial redun- dancy (Gallego et al. 2020 Gehrig Scaramuzza 2024 He et al. 2024a b). but challenge standard
machine learning (ML) algorithms, which typically demand tensor-like inputs because of the asyn- chronous and sparse nature the data (Sironi et al. 2018 Lagorce et al. 2016 Gehrig et al. 2019 Innocenti et al. 2021 Zubić et al. 2023). To bridge the gap between asynchronous data and syn- chronous ML algorithms, the asynchronous torponooss---syychooooooo (A2S) framework (Martin-Turrero Turrero et al.) 2024) has recently emerged. By designing efficient asynchronous encoder that recurrently inte- grates events into tensor-like features on an event-by-event basis and an on-demand sample strategy that feeds features to downstream MI. algorithms at flexible rates, this paradigm successfully lever- ages the fine-grained temporal information and inherent sparsity of events while benefiting from advances in standard ML.
However, existing A2S work is limited in model expressivity and feature generalizability. On one hand, the asynchronous encoder relies on preliminary models compromise for computational ef- ficiency considerations, which demands parallel training and equivalent recurrent inference. There- fore, the A2S method achieves suboptimal performance in complex tasks compared dense ap- proaches (Gehrig & Scaramuzza 2023) Gehrig et al. 2019 |eng et al.] 2023b; Fan et al. (025) with event images. On the other hand, the features are end-to-end learned supervised manner and thus are task-specific. This limits their generalizability across diverse downstream applications.
To design an A2S framework with more expressivity and generalizability, we take inspiration from the analogy between events and language (Soydan et al. 2024) and leverage recent advances in natural language processing (NLP). First, events and language are both organized in sequence. Sec- ondly, events record incremental visual changes, similar to words in sentence that incrementally contribute to the semantic meaning. These two similarities inspire us to treat each event as one token and leverage recent linear attention (LAA) architectures for efficient sequence modeling (Katharopou-

---

Published as a conference paper at ICLR 2026
Event-based Asynchronous
Readout
Tensor-based Synchronous
Async. events
Async. Encoder
Event-by-event updated Feature
Applicable to multiple downstream tasks
Event
Object recognition
camera
Tensor-based & Synchronous
Supervision
Object detection
Async.
Multiple handcrafted representations
events
Overview of the proposed EVA A2S framework. It has an asynchronous LA-based en- coder (top middle) producing event-by-event updated features for synchronous downstream tasks (right). These features are learned by suf--supevviioon (bottom middle) and are event-by-event up- dated during inference to be sampled demand by downstream tasks.
Figure 1:
los et al. 2020 Yang et al. 2023a Peng et al. 2023a Sun et al. 2023 and auto-regressive prediction (Radford et 2018 for feature learning.
Moreover, we also consider the distinctions between events and language, primarily in information density and spatial locality. One token in NLP has significant semantic meaning, whereas an indi- vidual event token offers limited information, gaining significance only when aggregated over time. This steers our focus towards the aggregated global information rather than isolated event instances. Additionally, events typically signify spatially localized changes, contrasting with language, which lacks inherent spatial attributes. This motivates modeling correlations primarily between spatially adjacent events to reduce complexity.
In this work, we introduce new A2S framework with a more expressive asynchronous encoder, which achieves event-by-event feature updating, and self-supervised method for generalizable feature learning (Figure I]. The asynchronous encoder built on the basis LA, which enables recurrent inference and parallel training. We propose to the matrix- hidden state (MVHS) introduced in LA as output to expand the model's expressivity on the aggregated global information and propose patch-wise encoding leverage the locality. For ease of implementation and better performance, our model is built on an open source high-performance LA architecture RWK (Peng et al. 2024), which implements MVHS and is stable train.
We propose a self-supervised learning (SSL.) method to learn generalizable features, which consists of two tasks: multi-representation prediction (MRP) and next-representation prediction (NRP). The former encourages the feature to learn diverse information from multiple handcrafted (converted) representations (Lagorce al. 2016 Sironi et al. 2018) comprehensively. and the latter, motivated by next-token prediction (NTP) in NLP (Radford et al., 2018), pushes the model beyond simple memorization. Our framework superior to the previous A2S work Mtartin-Turrent recognition tasks, and it is the first A2S framework, to the best of our knowledge, that successfully masters demanding detection tasks, achieving a 0.477 mAP on the Genl dataset. We name our framework EVent Asynchronous feature learning (EVA). Our contributions include:
al. 2024) on
1. An asynchronous LA-based encoder architecture derived from RWKV-6, enabling efficient event- by-event feature updating with improved expressivity.
A novel multi-tase task SSL method learning features generalizable to various downstream tasks.
A2S framework that combines the asynchronous encoder with self- supervised feature learning, outperforming the previous A2S work on recognition tasks and, for the first time, successfully masters demanding detection tasks with a 0.477 mAP the Genl dataset.
3. We present EVA,
RELATED WORK
2.1 ASYNCHRONOUS EVENT PROCESSING
To leverage the sparsity and high temporal resolution of event data, several recent studies have focused on asynchronous event processing. These include convolutional methods (Messikommer
2

---

Published as a conference paper at ICLR 2026
et al.
2020 Cannici et al. 2019), graph- models (Li et al. 202 Schaefer et al.
2022 Dalgaty
et al.
2023 Dampthoffer et al., 2025 Gehrig & Scaramuzza 2024| Li et al. 2025), attention- based
mechanisms (Kamal et | 2023). LSTM-based solutions (Cannici et al. [020;] Santambrogio al.,
2024), and point cloud approaches (Sekikawa et al. 2019 Martin-
et al. 2024). Convolu-
tional methods derive local activation update rules in standard (Cannici al. 2019 sparse (Mes- sikommer et al., (020) convolution layers. While they can leverage sparsity and reduce computation, they discard the temporal information. Graph-based methods transform events into ppatial-tempora graphs to capture both spatial and temporal information and leverage sparsity. However, graph- based methods show limitations in temporal accumulation (Dampthoffer al. 2025). An attention-based memory-augmented method was proposed in Kamal et al. (2023) to process batched events at each timestep. The LSTTMbaased Matrix-LSTM (Cannici et al. 2020) method learns and recurrently up- dates representation with pixel-wise LST' for downstream tasks. FARSE-CNN (Santambrogio ctt al. 2024) integrates LSTMs within the neurons of convolutional layers to capture temporll information and employs local update rules. Our work is most related to the A2S method point ALEKI-Transformer (Martin-Turrert et al.] 2024), which uses EventNet (Sekikawa et al. 2019) for asynchronous feature encoding. However, EventNet does not perform hierarchical learning and its expressivity is therefore limited. Our method follows the same A2S paradigm but improves expressivity with new architecture.
2.2 LANGUAGE-LIKE EVENT PROCESSING
Recently, several works studied processing raw events in token-wise way. Jiang et al. (2022) pro- cesses raw event sequence with Transformer directly, but is limited to the guadratic complexity of softmax attention on long event sequences. To solve this, [Schöne et al. (2024) and [Soydan et al. (2024) introduce state space models (SSMs) with linear complexity for asynchronous event process- ing. However, these methods do not discuss the similarity between events and language. Concurrent work (Fang & Panda] (2025) also models raw events with linear attention (Peng et al. (003a). How- ever, our work focuses the improvement of feature expressivity under the A2S paradigm, while Fang & Panda (2025) focuses on learning event representations like Word2Vec (Church 2017
2.3 SELF-SUPERVISED LEARNING ON EVENT DATA
Event data SSL can be broadly categorized into methods that learn from events only and those that incorporate other modalities. For event-only SSL, MEM (Klenk et al. (002) uses the masked image modeling method (Bao et al., 2021) to event images for pretraining. Other works (Barchid et al. Yang et al.] 2024) utilize contrastive learning with joint embedding architectures (Chen al,, He et al. 2020) on event images. These methods typically convert events to images and adapt established image SSL methods to the event domain. For SSL with other modalities, events are often paired with intensity images. In Yang et al. (002bb) contrastive pretraining is explored on event data synthetically converted from RGB images and their source images. EVRepSL learns representations
2023
2020
by exploiting the relationship between paired images and events (Qu et al. 2024).
3 METHOD
Our framework leverages the event-language analogy, and we first analyze their similarities and dis- tinctions. On one hand, events and language are similar in their (i) sequential nature and (ii) incre- mental manner. Events convey local intensity changes, similar to language tokens that incrementally build the context (Figure 2). On the other hand, key distinctions exist in their (i) information density and (ii) spatial locality. An individual language token has explicit semantics. In contrast, events only record pixel-level intensity changes and require temporal aggregation to become informative.
As illustrated in Figure 3 our A2S framework first transforms raw events into tokens and embeds them with their spatial and temporal attributes. These embeddings then asynchronously encoded into features by the RWKV-6-base encoder. The features are learned via SSL during training and generated event-by-event inference for real-time downstream applications.
3

---

Published as conference paper at ICLR 2026
Events
Sequential
Incremental
Language
Sequential
Still
P
The
black
circle
is
still
the
white
circle
Is
Move
Incremental
moving
Figure 2: Parallels between events and language: their sequential nature and incremental manner.
Event Tokenization & Embedding
b. Asynchronous Feature Update with Linear Attention
Events:
Embeddings
Event-by-event update
2-D Feature map
RWKV-6 blocks
MVHS layer
1-D 2-D
Event taken Tok (u. Timestamp
Spatial emb.
expansion
add
add
Temp. emb
Recurrent
Recurrent
c. SSL: Mult--Representation Prediction (MRP) & Next Representation Prediction (NRP)
Converted repr.
Pred. repr.
Repriipecific
Fi-1
R,
R
Head
R2
R2
Head
Exnr
6(+2
Fin
Figure 3: Architecture of the A2S framework: (a) Tokenizing and embedding events. (b) Asyn-
chronous encoding via LA. (c) MRP and NRP for self-supervised feature learning.
3.1 ARCHITECTURE: ASYNCHRONOUS ENCODER
Key challenges in designing an asynchronous event encoder include: (i) allowing recurrent infer- ence for real-time feature updates and parallel training on long event sequences, (ii) continuously encoding events with variable temporal intervals, and (iii) efficiently and expressively aggregating information into the feature. To address the first challenge, drawing from the event-language anal- ogy, find that recent linear attention (LA) supports both recurrent inference and parallel training. Building on LA, we incorporate temporal embedding, MVHS as output, and patch-wise encoding for the other challenges.
3.1.1 EVENT TOKENIZATION AND EMBEDDING
We define an event sequence as & = fe; = (t,, Vi. Pi.,,, 1,2....). where t. E R are the timestamp, coordinates, and polarity of events, with p (0, 1} indicating the two polarities. The spatial component of each event is tokenized using bijection Tok as:
Tok & N
(1)
(x. v.P) + x + W
where H, W denote the height and width of the spatial containing the event sequence. The vocabulary size is thus 2 x x W. This mapped token represents the spatial location of the event.
Each tokenized event e; (t, Tok (I,, Vi, P:)) is then embedded input vector æ; RD the encoder. For event tokens that represent spatial locations, we employ a learnable embedding layer on the tokens and get Emb (e:) RR!
Events are inherently asynchronous, so temporal information must be embedded the input. Con- sidering that event cameras generate events high temporal resolution, the absolute timestamp (in

---

Published as a conference paper at ICLR 2026
(us) quickly grows large in continuous operation; embedding the absolute timestamp would push the model beyond the range observed during the pretraining period, leading to the same kind of length- extrapolation failures in language models (Press et al. 2021 Su al. 2024 Chen et al. 2023). We therefore embed the time difference A4; 1 4--1 instead of the absolute timestamp, and use sinusoidal embedding (Vaswani al.) 2017). The final embedding is then formed by summing the spatial and temporal embeddings.
3.1.2 MODELING EVENT SEQUENCE WITH LINEAR ATTENTION
Background: Linear attention in RWKV-6. The asynchronous encoder is built on RWKV-6,
nigh-performance linear attention architecture for language modeling(Peng. et al.) 2024). For an input sequence (z, E Px 117 of length T, a single RWKV-6 block performs token mixing (TM) and channel mixing (CM).
In TM, inputs are first mapped to (ri, wi kG v2 E PXITT 1E and subsequently mixed by the rwke operator:
y1
diag
k + diag (u) k wij
ri EXI
(2)
where u E IR Dx1 is a Iearnable parameter. This ruko operator involves only element-wise prefix summations and products and therefore enables parallel training with scan (Martin & Cundy 2018). Equation can be expressed in a recurrent form:
S. = diag (w:) S + kz E IR DxD
y: = (S 1 + diag (u) k; r, E Dx1
Consequently, the hidden state S can be updated recurrently at inference upon the arrival of cach new event, allowing efficient training and recurrent inference on continuous long event sequences.
To better capture diverse information aspects, RWKV-6 uses multi-head TM. Each input is split into N heads of dimension Dnead = D/N, and the ruke operator acts on each head individually. The hidden state in multi-head TM is S R Prea Phea,
The CM layer transforms TM outputs using a ford-forward network, similar to Transformers Vaswann al. 2017). Appendix B provides details.
Matrix-value hidden states (MVHS) as output. NLP tasks typically involve sequence-to- sequence models mapping 1-D embeddings to 1-D embeddings. We propose train directly on the 2-D MVHS S, rather than the 1-D output y, and use S as the event feature, which offers several benefits. First, MVHS, the hidden state, naturally contains aggregated infor- mation. This aggregated feature is precisely what we want, instead of the mapping from em- beddings embeddings in standard auto-regressive pretraining (our attempt to predict the next single event embedding failed to converge). Secondly, MVHS provides an expanded memory of size N Dreai Dread without increasing model width D, thus improving the expressiv- ity of the learned feature and enabling a lightweight architecture for real-time event processing. Using MVHS as output can reduce model size by ap- proximately Dmodee/N versus 1-D outputs (Appendix G). Moreover, the 2-D structure of MVHS helps to learn fine finengrinedd spatial features, improving performance over prior 1-D features (Sekikawa et al. 2019 Martin-Turremt
Spatial partioo
Paxch wise encoding
Patch-wwise wiiso
presraining
Encoder
Head
Tarao
et al. 2024).
3.1.3 PIS ENCODING OF FEATURES
Contet
Downstroam
To exploit the locality of the event, we follow the prac- tice Martin & Cundy 2018) and use Patch-wise en- coding (PWE) for the feature. Specifically, for event camera with resolution (Heenoor W Wwenser) and patch size P. separate events to Hyensor W Weensor/P /2 sequences by their coordinates. As Figure Iillustrates, we encode a feature for each patch separately. With PWE, the model
tasks
Row events
Patch features
Feature
Figure 4: Patch-wise feature encoding. Events are partitioned and encoded by patch. The patch feature could be con- catenated for downstream tasks.
5

---

Published as a conference paper at ICLR 2026
size of the encoder can be reduced by factor of approximately the number of patches (# patches), which helps real-time inference (see Appendix G for discussion). Furthermore, since the asyn- chronous encoder trains fixed-size patches, this patch-wise encoding allows the application to event cameras with different resolutions.
3.2 SELF-SUPERVISED FEATURE LEARNING ON ASYNCHRONOUS EVENTS
We employ SSL train the asynchronous encoder. Since this training procedure is fask-irrelevant the resulting event feature is not tailored to certain task, thereby exhibiting better generalizability. The proposed SSI. method consists of two tasks: multi-representation prediction (MRP) and next representation prediction (NRP).
3.2.
MULTI-REPPRSSENTATION PREDICTION
For MRP, we force the encoded feature to predict multiple handcrafted (converted) event representa- tions (Lagorce et al. 2016 Sironi et al. 2018 Innocenti et al. 2021), such as time surface (TS), to learn a more comprehensive feature. For events & = {e; (fa, Tss Vr. P:). 1, }. we expect the asynchronous encoder Me parametrized by
count (EC) and
learn an went-by-event feature as F, = Mo(feshssi. Given K handcrafted representations {R k K
k==1 we predict these
K The objective is representations from F with K specific heads parameterized by C = {0 k==1'
K II (P = Mo (fe)))s)):0k)
arg max E, 0.0
(4)
where Pr means probability. MRP motivated by the insight that different handcrafted represen- tations capture different aspects of information about the events. They provide the required information for various tasks. Therefore, we anticipate that learning from multiple converted repre- sentations will yield feature generalizable diverse downstream tasks.
3.2.2 NEXT REPRESENTATION PREDICTION
Inspired by auto-regressive pretraining of language models, where models learn by next-token pre- diction (Radford al.) 2018), NRP forces the encoded feature predict handcrafted representations for future time interval. For future time window AT, the objective of NRP is
arg max E, 0
I Pr (R& (Lelt, < t(e) <t. ATH) |F, Mo (He;bis:):
(5)
We aim for the model to understand inherent motion patterns and go beyond just memorizing the history by learning to predict the future. In particular, individual events are ineffective as prediction targets due to their limited information and unpredictable noise. In contrast, converted representa- tions are more informative and noise-robust, offering a more reliable supervision signal.
EXPERIMENTS
4.1 OBJECT RECOGNITION
Datasets. To compare with previous A2S work (Martin-Turreero Turrero et al. 2024). we evaluate our EVA on two brent-based recognition tasks: action recognition (DVST28 Gesture (Amir et al.) 2017) and binary classification (N-Cars (Sironi et al. 2018)). DVS128-Gestne
provides human actions recorded by the DVS128 event camera, containing 1342 files of around 6 seconds from subjects under 3 different lighting conditions. N-Cars provides a benchmark for the binary classification task of car recognition, containing 24029 files of 100 ms recorded by the ATIS event camera.
Implementations. We present asynchronous encoder with embedding width D 128, RWKV- 6 blocks width D = 128, and MVHS layer as output to generate the learned feature. For PWE, reduce the latency of downstream classification,
P 16. On DVS128 128-Gestre D = for a 2 x shrink in height and width of the feature and use a lightweight 14-layer ResNet (He al. 2016) (referred to as ResNet-14) for classification. On N-Cars, however, since the samples

---

Published as a conference paper at ICLR 2026
cropped small sizes (Sironi et al. 2018). also present encoder with Diead 16 for full height and width feature. To maintain the feature size, we use only half of the theoretical channels of the MVHS output. The model implementation details are given in Appendix
On DVS128-Gesture, the encoder is pretrained on event sequences of length 8192 with batch size 32 and Adam optimizer (Kingma & Ba 2014) with learning rate 0.001. It takes 10 hours to train 100 epochs with an RTX 3090 GPU. On N-Cars, the encoder is pretrained with a sequence length of 512 with batch size 32 and learning rate 0.001. It takes hour to train 20 epochs on an RTX 3090 GPU. We use EC and TS for MRP and EC for NRP as the target. To balance the multi-task losses, we use uncertainty weights (Kendall et
2018 The details of loss items are given in Appendix
E
Metries. On DVSI28- Gesture, following previous work (Martin- Turrero et al. 2024], we evaluate the model on samples of length 8192 and report the accuracy on samples (sample accuracy, SA) and the accuracy after majority voting on all the samples of each file (file voting accuracy, FVA). We also report the complexity (MAC) and inference latency on the RTX 3090 GPU. On N-Cars, we report accuracy.
Results. On WVS128-Gesture. our model achieves 96.9% FVA and 92.9% SA (see Table [T. out- performing the best result previous A2S work (94. I 1% FVA and 84.6% SA) by 2.8% and 8.3% respectively. Compared with ALERT-Tr. (+RM), our EVA achieves better performance with smaller parameter size and complexity. ALERT-Tr. (+LMM) indeed has fewer parameters, but with much lower accuracy of 72.6% in SA, 20% less accuracy than ours. It is worth noting that EVA, with a lightweight ResNet classifier, has much lower inference time (1.5 ms) compared with both the LMM and RM. Our asynchronous encoder has larger inference latency compared with previous work because of its hierarchy learning architecture. However, it has a shorter processing time for an 8192 8192-length sample, which is about 100 ms in DVS -estuuee On N-Cars (Table 2), previous A2S work (Martin- Turrero et al., 2024) did not report the results with their best model (RM), only gave the accuracy of LMM to show its low complexity. Therefore, here we additionally compare our EVA with (i) dense event image- based methods (Peng et al.) 2023b) Klenk et al. 2024)
and (ii) methods that learn the representation/feature from raw events (Gehrig et al. [019]] Cannici al. 2020). In comparison with ALERT-Tr. (+LMM), our method has better accuracy and less classifier latency. Since the files in N-Cars are small number of events and the pretraining on this dataset is not sufficient, also present the result with features generated by an encoder pretrained on Genl for detection tasks (EVA-L in Table For a fair comparison with Gehrig et al. (2019) and Cannici al. (2020), follow their practice and use ResNet-34 pretrained on ImageNet (Deng iet al. 2009) as the classifier. Our model, with the encoder pretrained on Gen and ResNet-34, has an accuracy of 96.3%, higher than the methods with learned representations.
Table 1: Results on DVS128-Gesture Latency refers to the time to process sample of length 8192. We compare with previous A2S work ALERT-Tr. AMartin-Turrert al. 2024).
MAC per
Acc.
Model
# Params
Latency
Event
Sample
SA
FVA
ALERT-Tr.
1.41 M
1.22 M
8.83 G
5.8
84.6%
94.1% 1%
(+RM)
13.96 M
1.30 M
9.42 G
9.6 ms
ALERT-Tr.
0.04 M
0.0040 M
0.03 G
3.9 ms
72.6%
89.2%
(+LMM)
0.57 M
0.0074 M
0.06 G
6.0 ms
EVA (ours)
0.62 M
0.60 M
4.79 G
14.7 ms
92.9%
96.9%
(+ResNet-14)
2.83 M
0.28 M
1.82 G
1.5 ms
4.2 OBJECT DETECTION
Dataset. We evaluate work on the challenging automotive detection dataset Genl (De Tourne- mire et al.) (020), which includes 39 hours of data recorded by the Genl event camera with a sensor resolution of 304 240.
7

---

Published as a conference paper at ICLR 2026
Table 2: Results N-Cars. EVA is pretrained N-Cars, and EVA-L pretrained on Gen
Model
Event Repr.
Acc.
GET
Peng et al. 2023b
Event count + Token
96.9%
MEM
Kllekk et al.
2024
Event count 98.4%
EST Gehrig et al. 2019)
Learned 92.5%
Matrix-LSTM Cannic al. 2020
Learned 95.8%
ALERT-Tr. (+LMM)
Learned
85.6%
EVA (+ResNett14) (ours)
Learned
91.4%
EVA (+ResNet-14) (Dhead = 16) (ours)
Learned
91.6%
EVA-L (+ResNet-34) (Deead 16) (ours)
Learned 96.3%
Implementations. We present EVA of RWKV-6 blocks, one MVHS layer with = 128. We also present larger model with D 192 (referred to EVA-L). We patch size P = 16 and Dhead = 16 to keep the spatial resolution. Like N-Cars, we use only half of the output channels of the MVHS layer. For the downstream detection task,
RVT-B (Gehrig & Scaramuzza 2023). a lightweight state-of-ther tf-he--rrth (SOTA) Transformer-based synchronous backbone and YOLOX detection head (Ge al.) 202T) on the learned features. The model implementation details are given in Appendix D
We pretrain the encoder with sequence length T 2048 and batch size 64 on RTX 3090 GPUs. The learning rate is initially 0.001 with a decay of y 0.9 every epoch. We use EC and TS for MRP objectives and EC for NRP. The details are given in Appendix
Results. Existing A2S work did not report results object detection tasks. Therefore, here we compare our method with (i) synchronous dense methods and (ii) end-to-end asynchronous methods. The results shown in Table 3 Asynchronous methods have minimal latency but lower accuracy compared with synchronous methods. Our best model achieves 47.7 mAP, and improves the SOTA synchronous method (Gehrig & Scaramuzza 2023) with lower number of channels of the input feature (6 for our EVA compared with 20 for RVT-B). This is the first result by A2S methods on event-based detection tasks, demonstrating the effectiveness of our method.
Object detection performance Genl. A refers to end-to-end asynchronous methods. refers to synchronous dense methods.
Table 3:
Model
Type
mAP (%)
NVS-S (i et al.
2021
8.6
AEGNN (Schaeter al.
2022
A
14.5
Asynet Mcssikommer et al.
2020
16.3
FARSE- (Sattambrogio al.
2024
A
30.0
DAGr-L (Gehrig & Scaramuzza
2024)
32.1
ASTMNet (Li et al. 2022
46.7
HMNet-L.3 (Hamaguchi et 2023)
47.1
RVT-B (Gehrig & Scaramuzza
2023
S
47.2
GET (Peng et al.
20236)
47.9
EVA (+RVT-B) (ours) (D 128, D = 16)
A2S
47.5
EVA-L (+RVT-B) (ours) (D 192, Daead = 16)
A2S
47.7
4.3 TIMING EXPERIMENTS
RTX MACs. per event. We timed EVA models, implemented in Python and CUDA, 3090 GPU. We summarize the results in Table and the event rate (number of events per second) of the datasets in T For DVS128-Gestur and N-Cars, the event rates are both within 100,000 per second, lower than the throughput of the EVA models (at least 541,000 per second). For Genl with larger resolution, the event rate surpasses the throughput of our EVA-L model. Since patch-
8

---

Published as a conference paper at ICLR 2026
wise encoding (PWE) is used in our EVA encoder, the features of each patch could be encoded independently. Therefore, considering that the patch event rate of the Genl dataset is 2.17 K/s, much lower than the throughput of EVA-L, the real-time encoding on Genl could still be achieved.
Table 4: Throughput and MACs of EVA models.
Model
Param
MAC's per Event
Throughput (M/s)
EVA
0.6
0.60 M
0.611
EVA head 16)
0.6 M
0.61 M
0.560
EVA-L
1.4 M
1.31 M
0.541
Table 5: Event rates of different datasets.
Dataset
Camera
Resolution
Event rate (M/s)
Patch event rate (K/s)
DVS128-
DVS128
128x 128
0.058
0.90
N-Cars
ATIS
100x 120
0.039
4.07
Genl
ATIS
304 240
0.618
2.17
Latency vs. Context length. We evaluate the inference time with respect to input context length.
As shown in Figure []] the inference time is basically proportional to the context length (number
of input events). We could leverage temporal subsampling (Santambrogio et al.] 2024) or pooling (Soddan et al. 2024 to reduce MACs, which commonly used in asynchronous event processing.
Latency Context Length
EVAt
EVA (close 1bb
DVA-L
30
006
40
10
tockns)
Figure 5: Inference latency different sequence lengths.
Latency vs. Sensor resolution. For high-resolution event cameras (i.e., Gen with 1280 720 pix- els), the event rate would increase proportionally to the sensor size. Therefore, these high-frequency event sequences would challenge our EVA encoder. Theoretically, since EVA conducts PWE strat- egy, the features of each patch could be computed in parallel. The overall runtime is only determined by the longest computation time among all the patches, and thus only determined by the patch event rate. From Table [5) the patch event rate is stable across event cameras with different resolutions. Therefore, our method could, in principle, be applicable event cameras at any resolution, without introducing further latency.
4.4 ABLATION STUDY
We ablate architecture components (MVHS and temporal embedding) and SSL objectives to check their contribution. The effect of patch size is also investigated. The ablation is conducted on DVS128-Gesture following the implementations in the object recognition experiment. We report the loss items of SSL during pretraining and downstream accuracy the validation set.
Ablation of MVHS and temporal embedding. To remove MVHS, we keep the model width D 128 unchanged and let Dread in the output layer. The output becomes vector with shape

---

Published as a conference paper at ICLR 2026
128 1 x 1. The results shown in T Models without MVHS temporal embedding show a decrease in both FVA and SA, and also have a larger pretraining loss.
Table 6: Ablation of MVHS and temporal embedding.
MRP Loss
NRP Loss
Acc.
MVHS
Tempo. Emb.
EC (L)
TS (4)
EC(()
FVA (f)
SA (1)
0.366
3,16x10- 3
0.767
98.1%
94.7% %
1.870
4.56x10-
0.933
87.8%
81.1% %
/
0.826
9.34x10->
0.834
97.4%
94.1%
Ablation SSL objective. We remove the SSL objectives to show their contribution to the learned feature. Results in Table 7 show that removing the items of SSL decreases the performance of the model. One interesting thing is that learning only one representation, such as EC, will not contribute to smaller pretraining loss compared with learning all the representations. This indicates that learning one single representation could benefit from learning other representations.
Table 7: Ablation of objectives MRP and NRP.
MRP
NRP
MRP Loss
NRP Loss
Acc.
EC
TS
EC
EC (L)
TS (L)
EC(L)
FVA (t)
SA (t)
/
0.366
3.16x 10-3
0.767
98.1 %
94.7%
0.639
x.03x10-2
96.8%
94.4%
0.701
96.8%
91.7%
/
1.40x10-2
97.4%
91.6%
/
0.825
96.8%
91.8%
Effeet of patch size. We change the patch size from 16 to 128 (w/o patching in this case), while keeping the width of the encoder D 128 and the sequence length 7 = 8192 for different patch sizes. The results in Table @ show that a small patch size gives better accuracy. Patch size P = 128 gives a smaller pretraining loss because the spatial sparsity makes larger patch have more blank areas.
Table 8: Effect of the choice of patch size
MRP Loss
NRP Loss
Acc.
Patch Size
EC (4)
TS (1)
EC(L)
FVA (t)
SA (t)
16
0.366
3.16x10-3
0.767
98.1 %
94.7%
32
0.454
1.67 10-2
0.393
94.8%
91.0%
64
0.233
x.45x10-2
0.149
95.5%
89.6%
128 (w/o patching)
0.108
9.-8x10-3
0.073
97.4%
89.3%
CONCLUSION
We introduce A2S framework, EVA, for real-time event camera data processing. The frame- work consists of an efficient and expressive linear attention-base asynchronous encoder for event- by-event feature updating and an SSL method learn generalizable event feature. EVA out- performs the previous A2S work (Martin- Turrero et al. 2024), achieves remarkable performance event-based detection tasks, and learns generalizable teatures applicable in diverse downstream tasks, demonstrating its potential towards real-time event-based applications.
The limitations of work discussed in Appendix
10

---

Published as a conference paper at ICLR 2026
REPRODUCIBILITY STATEMENT
Code available https:: /gtt::b/gttubbbub 9ct/ Datasets: The data pre- processing steps are described in Appendix [ Experiments: All experimental settings, including hyperparameters and model architectures, are detailed in Section 4 and Appendix D The random seeds used for all experiments are fixed.
Code:
ACKNOWLEDGEMEET
This work was supported by STI 2030-Major Projects 20212D0200300 and High Performance Computing Center of Tsinghua University.
REFERENCES
Arnnn Amir, Brian Taba, David Berg, Timothy Melano, Jeffrey McKinstry, Carmelo Di Nolfo, Tapan Nayak, Alexander Andreopoulos, Guillaume Garreau, Marcela Mendoza, et al. low power, fully event-based gesture recognition system. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7243-7252, 2017.
Wes Baldwin, Ruixu Liu, Mohammed Almatrafi, Vijayan Asari, and Keigo Hirakawa. Time- ordered recent event (tore) volumes for event cameras. IEEE Transactions Pattern Analysis and Machine Intelligence, 45(2):2519 2522)2522,2222 2022.
Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. Beit: Bert pre-training of image transformers. arXiv preprint arXiv:2106. 08254, 2021.
Sami Barchid, José Mennesson, and Chaabane Djéraba. Exploring joint embedding architectures and data augmentations for self-supervised representation learning in event-based vision. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3903- 3912, 2023.
Marco Cannici, Marco Ciccone, Andrea Romanoni, and Matteo Matteucci. Asynchronous convolu- tional networks for object detection in neuromorphic cameras. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 0-0, 2019.
Marco Cannici, Marco Ciccone, Andrea Romanoni, and Matteo Matteucci. differentiable recur- rent surface for asynchronous event-based data. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Purt XX 16, pp. 136-152. Springer, 2020.
Shouyuan Chen, Sherman Wong, Liangjian Chen, and Yuandong Tian. Extending context window of large language models via positional interpolation. arXiv preprint arXiv:2306.15500 2023.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. International conference on machine learning, pp. 1597-1607. PmLR, 2020.
Kenneth Ward Church. Word2vec. Natural Language Engineering, 23(1):155-162, 2017.
Thomas Dalgaty, Thomas Mesquida, Damien Joubert, Amos Sironi, Pascal Vivet, and Christoph Posch. Hugnet: Hemi-spherical update graph neural network applied to low-latency event-based optical flow. In Proceedings the IEEE/VV Conference Computer Vision and Pattern Recog- nition, pp. 3953-3962, 2023.
Manon Dampfhoffer, Thomas Mesquida, Damien Joubert, Thomas Dalgaty, Pascal Vivet, and Christoph Posch. Graph neural network combining event stream and periodic aggregation for low-latency event-based vision. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 6909-6918, 2025.
Pierre De Tournemire, Davide Nitti, Etienne Perot, Davide Migliore, and Amos Sironi. A large scale event-based detection dataset for automotive. arXiv preprint arXiv:2001.0844 2020.
11

---

Published as a conference paper at ICLR 2026
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hi- erarchical image database. In 2009 JEEE conference on computer vision and pattern recognition, pp. 248-255. Icce, 2009.
Rui Fan, Weidong Hao, Juntao Guan, Lai Rui, Lin Gu, Tong Wu, Fanhong Zeng, and Zhangming Zhu. Eventpillars: Pillar-based efficient representations for event data. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 2861-2869, 2025.
Wei Fang and Priyadarshini Panda. Event2vec: Processing neuromorphic events directly by repre- sentations in vector space. arXiv preprini arXiv:2504. 15371, 2025.
Guillermo Gallego, Tobi Delbrück, Garrick Orchard, Chiara Bartolozzi, Brian Taba, Andrea Censi,
Stefan Leutenegger, Andrew J Davison, Jörg Conradt, Kostas Daniilidis, al. Event-based vision: A survey. IEEE transactions on pattern analysis and machine intelligence, 44(1):154- 180, 2020.
Zheng Ge, Songtao Liu, Feng Wang, Zeming Li, and Jian Sun. Yolox: Exceeding yolo series in 2021. arXiv preprint arXiv:2107.005. 084000 2021.
Daniel Gehrig and Davide Scaramuzza. Low-latency automotive vision with event cameras. Nature, 629(8014): 1033444))034-044 2024.
Daniel Gehrig, Antonio Loquercio, Konstantinos Derpanis, and Davide Scaramuzza. End-to-end learning of representations for asynchronous event-based data. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5633-5643, 2019.
Mathias Gehrig and Davide Scaramuzza. Recurrent vision transformers for object detection with event cameras. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 13884-13893, 2023.
Ryuhei Hamaguchi, Yasutaka Furukawa, Masaki Onishi, and Ken Sakurada. Hierarchical neural memory network for low latency event processing. In Proceedings of the IEEE/CVF Conference 2023.
Computer Vision and Pattern Recognition, pp.
Botao He, Ze Wang, Yuan Zhou, Jingxi Chen, Chahat Deep Singh, Haojia Li, Yuman Gao, Shaojie Shen, Kaiwei Wang, Yanjun Cao, et al. Microsaccade-inspired event camera for robotics. Science Robotics, 990))eaa1881 24, 2024a.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference computer vision and pattern recognition, pp- 770-778, 2016.
Kaiming He. Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9729-9738, 2020.
Weihua He, Junwen Zhu, Yongxiang Feng, Fei Liang, Kaichao You, Huichao Chai, Zhipeng Sui, Haiqing Hao, Guoqi Li, Jingjing Zhao, et al. Neuromorphic-enable video-activated cell sorting. Nature Communications, 10792, 2024b.
Simone Undri Innocenti, Federico Becattini, Federico Pernici, and Alberto Del Bimbo. Temporal binary representation for event-based action recognition. In 2020 25th International Conference on Pattern Recognition (ICPR), pp. 10426-10432. IEEE, 2021.
Bin Jiang, Zhihao Li, M Salman Asif, Xun Cao, and Zhan Ma. Event transformer. arXiv preprint arXiv:2204.05172. 2022.
Uday Kamal, Saurabh Dash, and Saibal Mukhopadhyay. Associative memory augmented asyn- chronous spatiotemporal representation learning for event-based perception. In The Eleventh International Conference on Learning Representations, 2023.
Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, and François Fleuret. Transformers mns: Fast autoregressive transformers with linear attention. In International conference on ma- chine learning, pp. 5156-5165. PMLR, 2020.
12

---

Published as a conference paper at ICLR 2026
Alex Kendall, Yarin Gal, and Roberto Cipolla. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7482-7491, 2018.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint 11X.66141 2014.
Simon Klenk, David Bonello, Lukas Koestler, Nikita Araslanov, and Daniel Cremers. Masked event modeling: Self-supervised pretraining for event cameras. In Proceedings of the IEEF/CVF Winter Conference on Applications of Computer Vision, pp. 2378-23888 2024.
Xavier Lagorce, Garrick Orchard, Francesco Galluppi, Bertram E Shi, and Ryad B Benosman. Hots: a hierarchy of event-based time-surfaces for pattern recognition. JEEE transactions on pattern analysis and machine intelligence, 19(7):134-11399 2016.
Dianze Li, Jianing Li, Xu Liu, Xiaopeng Fan, and Yonghong Tian. Asynchronous collaborative graph representation for frames and events. In Proceedings the Computer Vision and Pattern Recognition Conference, pp. 1655-1666, 2025.
Jianing Li, Jia Li. Lin Zhu, Xijie Xiang, Tiejun Huang, and Yonghong Tian. Asynchronous spatio- temporal memory network for continuous event- object detection. IEEE Transactions on Image Processing, 31:2975-2987, 2022.
Yijin Li, Han Zhou, Bangbang Yang, Ye Zhang, Zhaopeng Cui, Hujun Bao, and Guofeng Zhang. Graph-based asynchronous event processing for rapid object recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 934 943, 2021.
Eric Martin and Chris Cundy. Parallelizing linear recurrent neural nets over sequence length. In International Conference Learning Representations, 2018.
Carmen Martin- Martin-Turrero, Maxence Bouvier, Manuel Breitenstein, Pietro Zanuttigh, and Vincent Par- ret. trert-transformer: Bridging asynchronous and synchronous machine learning for real-time event- ovent-based ppatio-tempora data. arXiv preprint arXiv:2402.01393, 2024.
Nico Messikommer, Daniel Gehrig, Antonio Loquercio, and Davide Scaramuzza. Event-based asyn- chronous sparse convolutional networks. European Conference on Computer Vision, pp. 415- 431. Springer, 2020.
Garrick Orchard, Ajinkya Jayawant, Gregory Cohen, and Nitish Thakor. Converting static image datasets to spiking neuromorphic datasets using saccades. Frontiers in neuroscience, 9:437, 2015.
Bo Peng, Eric Alcaide, Quentin Anthony, Alon Albalak, Samuel Arcadinho, Stella Biderman, Huanqi Cao, Xin Cheng, Michael Chung, Matteo Grella, et al. Rwkv: Reinventing mns for the transformer era. arXiv preprint arXiv:2305. 13048, 2023a.
Bo Peng, Daniel Goldstein, Quentin Anthony, Alon Albalak, Eric Alcaide, Stella Biderman, Eugene Cheah, Teddy Ferdinan, Haowen Hou, Przemyslaw Kazienko, et al. Eagle and finch: Rwkv with matrix-valued states and dynamic recurrence. arXiv preprint arXiv:2404.0m 05892, 3, 2024.
Yansong Peng, Yueyi Zhang, Zhiwei Xiong, Xiaoyan Sun, and Feng Wu. Get: Group event trans- former for event-based vision. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 6038-6048, 2023b.
Ofir Press, Noah A Smith, and Mike Lewis. Train short, test long: Attention with linear biases enables input length extrapolation. arXiv preprint arXiv:2108. 12409, 2021.
Qiang Qu, Xiaoming Chen, Yuk Ying Chung, and Yiran Shen. Evrepsl: Event-stream representation via self-supervised learning for event-based vision. IEEE Transactions on Image Processing, 2024.
Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. Improving language under- standing by generative p2e-training.(2118)) 2018.
13

---

Published as a conference paper at ICLR 2026
Henri Rebecq, René Ranftl, Vladlen Koltun, and Davide Scaramuzza. High speed and high dy- namic range video with an event camera. IEEE transactions on pattern analysis and machine intelligence, 4366):196-11980, 2019.
Riccardo Santambrogio, Marco Cannici, and Matteo Matteucci. Farse-enn: Fully asynchronous, recurrent and sparse event-based cnn. In European Conference on Computer Vision, pp. 1-18. Springer, 2024.
Simon Schaefer, Daniel Gehrig, and Davide Scaramuzza. Aegnn: Asynchronous event-based graph neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.
2022.
Mark Schöne, Neeraj Mohan Sushma, Jingyue Zhuge, Christian Mayr, Anand Subramoney, and David Kappel. Scalable event-by-event processing of neuromorphic sensory signals with deep state-space models. In 2024 International Conference on Neuromorphic Systems (ICONS), pp. 124-131. IEEE, 2024.
Yusuke Sekikawa, Kosuke Hara, and Hideo Saito. Eventnet: Asynchronous recursive event process- ing. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 3887-3896, 2019.
Amos Sironi, Manuele Brambilla, Nicolas Bourdis, Xavier Lagorce, and Ryad Benosman. Hats: Histograms of averaged time surfaces for robust event-based object classification. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1731-1740, 2018.
Taylan Soydan, Nikola Zubić, Nico Messikommer, Siddhartha Mishra, and Davide Scaramuzza. S7: Selective and simplified state space layers for sequence modeling. arXiv preprint arXiv:2410.03444. 2024.
Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: En- hanced transformer with rotary position embedding. Neurocomputing, 568: 127063. 2024.
Yutao Sun, Li Dong, Shaohan Huang, Shuming Ma, Yuqing Xia, Jilong Xue, Jianyong Wang, and Furu Wei. Retentive network: A successor to transformer for large language models. arXiv preprint arXiv:2307.08621, 2023.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural informa- fion processing systems, 30, 2017.
Songlin Yang, Bailin Wang, Yikang Shen, Rameswar Panda, and Yoon Kim. Gated linear attention transformers with hardware-efficient training. arXiv preprint arXiv:23/2.06633, 2023a.
Yan Yang, Liyuan Pan, and Liu Liu. Event camera data pre-training. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10699-10709, 2023b.
Yan Yang, Liyuan Pan, and Liu Liu. Event camera data dense pre-training. In European Conference on Computer Vision, pp. 292-310 Springer, 2024.
Nikola Zubić, Daniel Gehrig, Mathias Gehrig, and Davide Scaramuzza. From chaos comes or- der: Ordering event representations for object recognition and detection. Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 12846-12856, 2023.
A EXTENDED EXPERIMENTAL RESULTS
A.1 OBJECT RECOGNITION
We provide object recognition experiments static-captured event camera datasets. We evaluate the EVA feature on N-Caltech101 (Orchard et al., (015)) dataset, which created by moving an event camera that is focused dataset consists 100 object classes and an additional background class. Since the dataset does not provide official train/test split, we follow the split EST (Gehrig et al. 2019) for fair comparison.
screen displaying the original Caltech101 dataset samples. The
14

---

Published as a conference paper at ICLR 2026
The results shown in Table Our EVA method outperforms all the asynchronous methods and has competitive accuracy compared with synchronous methods. Our method surpasses EST (Gehrig et al. 2019) and MatrixLSTM (Cannici al.] 2020), which learn representations in supervised, end-to-end manner. E2VID reconstructs gray-scale images and introduces an extra latency (Rebecq al. 2019). Our EVA method, however, does not lead to further latency and only shows marginal 0.3% drop in accuracy. This demonstrates that our EVA feature could also be used on the challenging static-captured datasets.
Table 9: Object recognition results N-Caltech101, A. refers asynchronous. S. refers syn- chronous.
Model
Type
Acc.
HOTS Lagorce et al.
2016
A.
21.0 %
NVS-S LI al. 2021
67.0
AEGNN [Schacter al.
2022
A.
66.8 %
Asynet (Caniic ct al.
2019)
A.
74.5
FARSE-CNN CNN (Santambrogio al. 2024)
68.7 % A.
EST (Gehrig al. 2019)
83.7
Matrix-LSTM (C (Cannici et al. 2020
S.
84.3 %
Pillar Fan al.) 2025
85.3
E2VID Rebecq et al. 2019)
S.
86.6 %
EVA-L (ours)
A2S
86.3 %
A.2 OBJECT DETECTION
To better demonstrate the expressivity and low-latency advantage of the learned asynchronous EVA feature, here we compare our EVA method with other represenpis methods under an identical neural network backbone (Swin Transformer V2). We compare both accuracy and representing latency, which refers to the time required for computing the representation when new event arrives Fan et al. 2025). The results are shown in Table 10
Table 10: Comparison of different event representations with an identical Swin V2 backbone on the Genl detection task. Representing latency refers the time required for computing the representa- tion when new event arrives.
Method
mAP (%)
Asynchronous
Representing latency (ms)
Voxel al.
2025
39.5
16.3
ERGO-12 Zubic et al.
2023
50.4
> 10
Pillar Fan et 2025)
53.1
11.4
HOTS Lagorce TORE
2016)
49.0
1.0
Baldwin et al.
2022
43.6
< 0,1
EVA-L (ours)
49.7
< 0.1
EVA (ours)
49.0
< 0.1
Synchronous representations provide better performance, but with additional latency. Our EVA method outperforms all the asynchronous representations and maintains its low-latency characteris- tic. This further demonstrates the advantages of our method in loteney applications.
A.3 TIMING EXPERIMENTS
We also evaluate the accuracy with different event data downsampling rates. Considering that in real-world applications, an event camera could generate events at different rates depending on the scene dynamics, and in case of high event rates which surpass the throughput of the EVA encoder, critical to understand how it performs under such conditions. We conduct experiments on DVS128 DVS128-Gesture Gesture dataset, and downsample the raw event sequence lower event rates. We evaluate
15

---

Published as a conference paper at ICLR 2026
the recognition accuracy and throughput of our EVA method. We freeze the pretrained weights of EVA encoder. The results are shown in Table 11 For shorter sequence length, the throughput correspondingly increases, while the accuracy will decrease due to the information loss from down- sampling. However, even with a 8x downsampling, our accuracy (95.1 %) still higher than the baseline (ALERT-Tr, 94,1%) with higher throughput (4888 K/s 1412 K/s).
Table 11: Accuracy and throughput under different event rates with downsampling.
Model
Length
SA.
FVA.
Throughput (K/s)
8192
92.9
96.9
611
4096
91.8
96.2
1222
EVA (ours)
2048
91.3
95.3
2444
1024
90.0
95.1
4888
512
88.4
93.4
9776
ALERT-Tr. (+RM)
8192
84.6
94.1
1412
ALERT-Tr. (+LMM)
8192
72.6
86.2
2100
B ARCHITECTURE DETAILS OF THE ASYNCHRONOUS ENCODER
RWKV-6 details. As described in (Peng et al. 2024), one RWKV-6 block is composed of token mixing (TM) and channel mixing (CM). For input sequence {x; E P of length T, TM first conduct a data-dependent linear interpolation (ddlerp) between æ; and æ; by
|/=1
lorayr.k...ff (a.)) (r.kwg)) tanh (x, A (r B (r.ino))
(6)
ddlerp((.k,.g) (Fi, 34 mi + (at-1 a;) @ lora (2; + (0:-1 v.) 3 pc):
where ,t Ar.kee E IRD are learnable parameters and A (nk.r.g) E BDD (r.k.og) Den x are trainable weight matrices. The vectors in Equation are calculated as
T: ddlerp, 2;-1 W,
k; = ddlerpa (x;, 1;-1) Wk:
11 ddlerp, (20, (0-1);;11
(7)
9. ddlerp, (2;; 1:
d, =
1));
w. exp(- exp (d.))
where W (n. E DRD A E DD B RDxxD trainable weight matrices.
Equation we have y: R. Dx1, The output TM given as
o concat (SiLU(g:) LayerNorm(y.) W. Px1
(8)
In CM, for inputs {at E RR Ji=1 linear interpolation (lerp) is first conducted lerp oe , (2t, a/-1) = x¹ + (x(_1 x 2()((r
(9)
where (rr))e))) R is trainable parameter. CM is conducted
r/ lerp, W, E ERD,
ky lerp (z), W E Pm..
(10)
J ReLU(K)) R
of o(r)) @ ( R.E
where W., E DDD W E Dxmm, W., E D trainable matrices.
MVHS layer. The MVHS layer is modified from the TM layer in RWKV-6. For input sequence ( RDTT Ji=1 of length 7, MVHS layer first maps them into k; Ut, w, Equation Then, given
16

---

Published as a conference paper at ICLR 2026
number of heads N and dimension of head Deaa D/N, the output S of the MVHS layer given
as
w wi (h.read (h + Dhead - 1| R Dred
k ki IheDe (h. + +) 1)Dned
1] E IR Doeu
0 [hDre (h + D)
1] E R Dacal
(11)
S
Σ diag
1 (u,
R Poza Daat 0,,,,,.0..... NN-
j=t+1
S. ";=conca(((")) Jh=
R
N' Droud Doest
This is equivalent directly using the multi-channel MVHS of the TM in RWKV-6 as the output. The hidden state preserves the aggregated information of the input events, which is precisely what we want for the event feature. Moreover, the multi-channel matrix shape matches the multi-channel image-like handcrafted event representations and could convey expressive fine- tieeggained spatial infor- mation. Therefore, we think using MVHS as output is a key towards expressive event features. The data flow of the EVA framework, from events the targets, is summarized in Figure
Event sequence
Targets C, P.P)
Tokenization Embedding
Embeddings (T.
Head
Conv (xM)
RWKV-6 Block (xN)
Features (T. DH.. DIH)
Token Mixing
MVHS Layer
Channel Mixing
Hidden states (T. DIH, DM))
Embeddings (T,
K, (T.
(Serqannca length: (Embedding with) (A Heads) C (Target channels) (Patcs win))
Figure 6: Data flow of the framework.
C DATASET PREPROCESS
has 11 classes of human actions of 29 different users. The official split the dataset uses users 0 23 the train split and users 24 29 as the test split. We follow this method and use users 21 to 23 in the original train split as a validation set. The pretraining conducted on the new training (user to 20) and validated on the val set (user to 23). We report the test accuracy of the best model on the validation set.
DVS128-Gesture.
For pretraining the model on sequences of length 8192, we first split all the files into patches of patch size p = 16. Then, we slice each patch into sequences of length 10240 with stride of 8192 as samples. In each sample, the first 8192 events are the inputs to the model, and the last 2048 events used generate the NRP objectives. We save the samples in the data format of At, x, y, p and convert them into uint 16 to save storage. The difference of timestamps At may be clamped to the numeric range of uint16, but this happens seldom and has little influence the experiments.
N-Cars. N-Cars has files duration 100 ms. Each file has around 4000 events. After splitting the files into patches of size 16 x 16, the average length of each patch is around 200, which is quite small. Therefore, on this dataset, we split the patches into samples of length 768 with a stride of 512. The first 512 events inputs, and the last 256 events used for NRP objectives. Following
17

---

Published as a conference paper at ICLR 2026
(Gehrig al. 2019 Cannici et al. 2020 samples are saved in the same format as DVS128-
20% of the training set as the validation split. The
Genl. We first filter out all the hot pixels caused by event camera sensor errors in the dataset. For every 10 ms in the file, the events in a pixel are deleted if they activate more than 40 times (we check the normal files and find that this value is no more than 10 regularly). This is similar to limiting the maximum value in the event counts in the synchronous method (Gehrig & Scaramuzza (023). We use the official train-val-test split of Genl and split the files into patches of size 16 x 16. Since Genl is a large dataset with sufficient events for pretraining the model, we do not use all the events for pretraining one epoch. Instead, we uniformly sample a slice of length 4096 from each patch. The first 2048 events are inputs, and the last 2048 events are left for NRP objectives. For pretraining, we save the patches in the same format as WVS128-Gestuun
For training, all the generated features are multiplied by a factor of 1/8, converted to their absolute value of data type uint8 to be consistent with the numerical range of the original inputs of RVT (0 to 20). We do this because we find that if we use the original float data type of the features, there will probably be numerical NaN error caused by the YOLOX head (Ge et al. 202T). This could also help save storage.
D IMPLEMENTATIONS
Three asynchronous encoders are presented in the experiment. In the experiments on DVS128- Gesture, we present an encoder (EVA) with 3 RWKV-6 blocks and I MVHS layer with D 128, Dhead = 8, N = 16, Dm. = 256, Disra = Dw = 16. The shape of the MVHS output is (16 x 8 x 8). To predict the representation of shape (* x 16 x 16),
a ResNet head with
basic ResNet block of width 64, I Conv Transpossess (in = 64, out = 32, Ks = 4, stride = 2) to upsample the 8 8 hidden to 16 x 16 and 1 basic ResNet block of width 32. We use one 1 x convolution layer as output. In the experiment on N-Cars, except for the encoder (EVA) on DVS128- Gesture, we also present an encoder with N 8, Dread = 16 (referred to EVA (Dnead 16) in the main context) since the spatial resolution of files in N-Cars is already cropped and small in spa- tial resolution. Using a shrunk hidden state as feature could indeed help reduce downstream tasks inference latency, but will decrease the accuracy (see Table 2) To keep the same total size of the feature, we use only half the channels in the MVHS layer, that is N = 4, Dreat = 16. The shape of the MVHS output is 4 x 16 x 16. In this case, since there is no need for upsampling, we use a ResNet with I basic ResNet block of width 32 as the head. In the experiment on Genl, we also present larger encoder with D 192, Dnead = 16, N = 8, Dim 384 (referred to as EVA-L (D = 192, Dneat = 16) in the main context) in the RWKV-6 blocks and N = 6, Dread = 16 in the MVHS layer. The same ResNet head N-Cars is used for pretraining.
For initialization, we use the official initialization method in RWKV-6 for the encoder and Kaim- ing initialization for the ResNet. The asynchronous encoder, including the RWKV-6 blocks and MVHS layer, is trained with bfloat16 precision as the default value in the original implementations in RWKV-6
In the recognition task on DVS128- 28-Gest we use a 14-layer 14) as the classifier by re- moving the last stage of the original ResNet-18 (He et al. 2016) to reduce inference latency (see Table 12). Interestingly, R4 has a higher accuracy on the validation set with a smaller pa- rameter size and lower latency. We think this is because smaller model could help overcome over- fitting. Despite this, we find that 18 could also outperform previous work (Martin-Turrery jtt al. 2024). We use batch size 32, Adam optimizer with learning rate 0.001, mixup augmentation, and train 200 epochs on a single RTX 3090 GPU with a decay of learning rate 7 0.8 every 20 epochs. In the experiments on N-Cars, we follow (Cannici et al. 2020; Gehrig et al. 2019) to use a
Table 12: Classification results on DVS128-Gesture with ResNet-18
Model
Classifier
# Params
Latency
Val. acc. (SA)
Test acc. (SA)
ALERT-Tr.
RM
13.96 M
9.6 ms
84.6%
EVA
Re
11.28 M
2.0 ms
92.8%
92.0%
EVA
ResNet- 14 (used)
2.83 M
1.5 ms
94.1%
92.9%
ResNet-34 pretrained on ImageNet as the classifier. Following the train settings of previous work,
18

---

Published as a conference paper at ICLR 2026
batch size of 64, Adam optimizer with constant learning rate of 0.0001, random flip as data augmentation, and train 100 epochs on an RTX 3090 GPU. In the experiments on Genl, we the default training setups of ( & Scaramuzza 2023 for a fair comparison. We use batch size 8, learning rate 2e-4, and train 400,000 steps on a single A800 GPU. The training takes about 2 days.
E PRETRAINING OBJECTIVES
We use event count (EC) and time surface (TS) as the pretraining objectives, Consider events & = {e; (ti, Fi. ye, p;) and patch size P, EC of given time window [ta, te] is constructed as channel image of shape 2 x P b
EC(p, 2.
Σ
w.w.::.ww-.................. E R2xPxP.
(12)
and TS is calculated
tr TS(,, max exp( to -Tr x ) -a= E R2XPXP
(13)
where + >> O is a time constant.
Table 13: Objectives for the SSL. For EC in MRP
MRP
NRP
Dataset
EC time window (ms)
TS 7 (ms)
EC time window (ms)
DVS128-Gestue
100
100
20
N-Cars
100
100
20
Genl
50, 25, 10, 5
200
10
Calculating the loss of the predictions and targets every individual event would demand prohibitive GPU memory. mitigate this, we divide the event sequence into chunks, and calculate the loss only at the end of each chunk. For DVS128-Gesture Cars and Gen1, we use chunk length Tehunk = 16.
Gesture dataset, use chunk length Tehunk 512. For N-
The objectives in the experiments TS r because we find it could help converge at the initial period pretraining to decrease the time required to pretrain the large dataset.
given in Table 13 On Genl, multiple ECs and larger
Table 14: Effect of SSL objectives
MRP
NRP
MRP Loss
NRP Loss
Acc.
EC (ms)
TS (ms)
EC (ms)
EC
TS
EC
FVA
SA
100
100
20
0.366
3.16x10
0.767
98.1%
94.7%
100 - 50
100
20
0.200
2.33x10-3
0.764
95.5%
94.3%
100 - 200
100
20
0.807
3.60x10-3
0.767
95.5%
94.3%
100
100 - 50
20
0.398
2.39x10-3
0.766
98.1%
94.1%
100
100 + 200
20
0.374
3.78x10-3
0.780
96.2%
94.4%
100
20 -
0.389
3.29x10-3
0.364
97.4%
94.2%
100
100
20
40
0.353
2086x10-3
1.698
98.7%
94.3%
For better understanding of the effects of the SSL objectives, here we give a study on DVS128- Gesture. The results given in Table We change the SSL objectives and find that the validation set accuracy (SA) is not heavily affected. The optimal configuration of the hyperparameter has not been investigated yet, and hope leave it for future study.
19

---

Published as a conference paper at ICLR 2026
F VISUALIZATION
We visualize the bandcrafted representations for SSL pretraining, the predicted results of the pre- training target, and the learned feature in Figure 7 The representations and results are generated on DVS128-Gesture
Figure (a) provides the visualization of handcrafted representations (targets in SSL), and Figure (b) shows the visualization of the reconstructed representations (predictions in SSL). The compar- ison between Figure 7 (a) and (b) demonstrates that during SSI., our neural network heads could consistently predict the corresponding handcrafted representations (EC, TS) with high similarity from the learned feature map. This shows that the learned features contain adequate information to reconstruct the handcrafted representations, and further shows that the EVA encoder is able cap- ture the majority of the spatial information from raw individual events. Additionally, the predicted EC (NRP) shows favorable consistency with its ground truth targets, demonstrating the success of predicting future values from the learned feature map. This indicates that our model is capable of learning short-term motion patterns rather than merely memorizing the inputs.
Figure 7 (c) provides the visualization of the learned feature map. Each figure is one channel of the MVHS layer output, and is a concatenation of multiple patches. Qualitatively, several channels of the learned feature map show patterns similar to the handcrafted representations, but with dif- ferent windows temporal accumulation (several channels preserve more past events, while others only maintain recent events). Besides, some channels show different patterns compared with the handcrafted representations. We leave a comprehensive quantitative analysis of the dynamics of the learned feature maps for further study.
G DETAILS OF THE MVHS AND PATCH-WISE ENCODING TO REDUCE MODEL SIZE
Assume that we keep the size of the learned feature unchanged. For original vector-value output, the output size is D, and the model parameter size is around ((D²) (since the majority of the parameters come from the matrix). For MVHS output, the output size N x (D' (N) x (D' (D) with model size (D)?). Let D = N (D'/N) x NN), we have O(D))/O(D²)) D/N EVA, this factor is 8 or 16.
Dread In our
For patch-wise encoding, assume that learn feature that is proportional to the size of the ob- jective representations. Consider an event camera with resolution (H. W) (assume H = W for simplicity) and patch size P, the number of patches # patches is H2 H/2/p2. Given the shape of the learned feature as (N x D X D), if we use it to predict the objective representations of full size, which is (* H H), then we have D H and the model size is around O(112). How- ever, if we use the learned feature to predict the representations of patch, which has the shape of (* P P), we have D P. and the model size is around O(P2). Therefore. the model size decreases O
= # patches times. In the experiment on DVSI28 2eGteree we have H W 128 and P = 16. This factor 64. On Genl with H 240, W 304 and 16, we have # patches = 285.
H LIMITATIONS
Our work is limited in two aspects. First, while our EVA framework is evaluated on publicly avail- able datasets recorded real-world settings, especially automotive driving scenarios, full explo- ration of its utility in practical applications requires implementing the models on device-side, such as on FPGAs, and testing the whole framework actual deployment scenarios. These aspects are not involved in the current study. Secondly, the models trained are relatively small in parameter size compared to pretrained large language models (LLMs) with billions parameters. Consequently, the potential of self-supervised pretraining could not be fully exploited in our study. The parame- ter size of the asynchronous encoder is constrained by the requirement for real-time processing of www---epooa----olllle
events. We aim to address this limitation in the future by decreasing the per-event computational complexity, thereby enabling the design of larger pretrained models capable of asynchronously processing events in real time.
20

---

Published as a conference paper at ICLR 2026
(a)
(b)
(8)
(R)
(MRP)
(MRP)
(NRP)
Learned regresentations
Visualization of 4 different gestures on DVS128-GER 122-Geeturee (a) Ground truth of the hand- crafted representations. From left to right: EC, TS, and EC in the future. (b) Neural network head predicted handcrafted representations in the pretraining stage. (c) The learned features. Each figure one channel of the MVHS layer output.
Figure 7:
21

---

Published as a conference paper at ICLR 2026
I COMPLEXITY ANALYSIS
Each update of the EVA features requires (M) computations, where M is the parameter size of the EVA encoder. Therefore, after sequential length of T, the total MACs is proportional the input length 7' and the model size M as (TTM). Here we summarize the complexity of other weentreppreenniing methods in Table T5| For EST. each time a new event arrives, the representations should be recomputed. The complexity is proportional to the number of events involved in the model size M, time window L. and number of channels of the EST representation C. For MatrixLSTM with temporal bins, the complexity is (LLM), which serves synchronous representation. For the Voxel grid, the complexity ( HWC). and for ERGO, the complexity is
Table 15: Complexity and estimated FLOPs of different representations. We consider L = X
10 H = 720, W 1280 event sequence the 1Mpx dataset.
Method
Asynchronous
Complexity per Event
Estimated FLOPs per Event
EVA-L
((M)
1.4 M
ALERT-Tr (RM)
(M))
1.2 M
EST
O HWC)
1152 M
MatrixLSTM
((LMC)
> 100
Voxel grid
O HWC)
11 M
ERGO-12
+
11 M
J
LLM USAGE
Large language models (LLMs) are used to improve the writing of the manuscript, including trans- lating languages, correcting grammatical and spelling errors, and improving sentence clarity.
22


Reviews:
[{"review_id": "FhhSdxAaGO", "reviewer": "Reviewer_q375", "summary": "This paper introduces EVA, an asynchronous architecture for event camera data. The key idea is to treat streams of visual events as analogous to language tokens, each carrying incremental information about a scene and to employ linear attention architectures for scalability. The encoder maintains a hidden state that aggregates spatial and temporal structure while supporting fast processing.", "questions": "Why is your latency higher when more parameters are used in Table 1?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "* Tackles a timely challenge as it is yet unclear how to process event camera inputs\n* Interesting ideas to use a token based approach with a linear transformer\n* The self-supervised aspect with the multi-representation training is seemingly novel and interesting", "weaknesses": "Overall, the major concerns I have with this work are that the results are not very compelling.\n\nThe authors only evaluate two task (object detection and recognition). Given the authors are proposing a new architecture, it would be good to have at least another slightly different task to demonstrate the robustness of their architecture is not clear. \n\nThe proposed approach achieves does not achieve the best accuracy or latency, but is another tradeoff point in the space. \n\nIf the key benefits are the fast processing, then it would be good to have a more thorough analysis of where the efficiency benefits come from and whether real-time performance can actually be achieve with their approach on realistic GPUs for edge computing.\n\nThe effectiveness of the approach with self-supervised learning using handcrafted representations is not convincing as the method to have it learn the right representations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GTB2ogIq5r", "reviewer": "Reviewer_4uAK", "summary": "This paper introduces EVA (EVent Asynchronous feature learning), a novel Asynchronous-to-Synchronous (A2S) framework for learning expressive and generalizable representations from event streams. The key idea is to treat events as “language tokens” and borrow concepts from language modeling, including linear attention and self-supervised feature learning, to construct a unified A2S architecture. EVA demonstrates strong results on multiple benchmarks—DVS128-Gesture and N-Cars for recognition tasks—and becomes the first A2S framework capable of performing detection, achieving 47.7 mAP on the Gen1 dataset.", "questions": "How is the number of events \\(N\\) in MVHS determined for different datasets, and how does it affect performance and computational cost?  \n\nThe motivation for selecting Event Count (EC) and Time Surface (TS) as the self-supervised prediction targets should be clarified, as well as whether other representations have been considered.  \n\nAlthough EVA avoids event accumulation, Table 1 shows an additional latency of 14.7 ms. Comparisons with other learnable event representations such as ERGO (“From Chaos Comes Order”) and EventPillars (“Pillar-based Efficient Representations for Event Data”) under identical backbones would better position EVA’s efficiency and expressivity.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The proposed framework is well-structured and systematically designed. The overall pipeline is coherent, covering event tokenization, temporal-difference encoding, MVHS feature generation, and self-supervised tasks in an end-to-end manner.\n\nThe experimental evaluation is comprehensive; the paper validates the generality of EVA across several datasets and tasks, includes meaningful ablation studies, and provides detailed experimental configurations.\n\nThe conceptual alignment between event representation and language modeling is novel and promising, opening opportunities for future research on cross-modal modeling between event-based vision and sequence learning.", "weaknesses": "In the supplementary material (Fig. 6(c)), it is unclear whether the visualized features correspond to the MVHS outputs. Section E. Visualization lacks a clear description or interpretation of these results.  \n\nWhile performance metrics on N-Cars and Gen1 are reported, latency, throughput, and scalability analyses are missing. Since A2S methods can, in principle, operate at arbitrary event sampling frequencies, it is important to analyze how performance and efficiency vary with sequence length.  \n\nThere are also minor presentation and clarity issues. The term “#patches” appears in Section 3.1.3 without prior definition and only reappears in the supplementary material. The notation \\(P_r\\) in Eqs. (4) and (5) is undefined, and Table 2 is never cited in the main text.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fulSaYvBV8", "reviewer": "Reviewer_n32k", "summary": "This paper presents EVA, a novel asynchronous-to-synchronous (A2S) processing framework for event cameras. EVA builds an asynchronous encoder based on a linear attention mechanism and leverages self-supervised learning to learn generalizable event representations. It significantly outperforms existing A2S methods across multiple event-based vision tasks, including object recognition and object detection. Notably, EVA achieves 47.7 mAP on the Gen1 detection dataset, and demonstrating its strong representation capability and generalization across complex tasks.", "questions": "1. The fitting targets during self-supervised training in this paper are EC and TS. Ideally, after training is completed, the proposed method could become some kind of weighted mixture of EC and TS. So why not directly use EC and TS? For example, in the experiments presented in this paper, what would be the performance change if the input representation were replaced with EC and TS?\n\n2. Integration of events and RGB frames is an emerging trend. Could this framework be extended in the future to design a joint asynchronous representation that fuses both modalities? It would be interesting to hear the authors’ thoughts or potential directions on this.\n\n3. Introducing NLP-inspired approaches into event representation is a novel idea. What specific advantages does this bring compared to conventional designs? Could the authors provide a more theoretical explanation or justification for this perspective?\n\n4. Looking ahead, could this asynchronous representation technique be incorporated into event-based multimodal large language models [2]? This would be an exciting direction for exploring the intersection between event-based vision and multimodal reasoning.\n\n[2] EventGPT: Event Stream Understanding with Multimodal Large Language Models, CVPR 2025.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1. The paper draws an insightful analogy between event streams and language sequences, introducing a linear attention mechanism and self-supervised learning to propose a novel and efficient paradigm for asynchronous event processing.\n\n2. The authors conduct extensive evaluations on multiple public datasets (DVS128-Gesture, N-Cars, and Gen1), covering both recognition and detection tasks. The results are convincing and demonstrate clear advantages over various baseline methods.\n\n3. The manuscript is well written and logically structured, with detailed figures and tables, and a comprehensive appendix that further supports the work.", "weaknesses": "1. As an A2S framework, a key question is how to achieve better asynchronous event representation. While I acknowledge that the authors have already included sufficient baseline comparisons, I wonder whether they have considered fixing the task-specific backbone and varying only the event representations, as described in Section 2.1. For example, in the object detection task on Gen1, one could fix a backbone (e.g., RVT-B) and compare different existing representation methods (e.g., time-surface, Matrix-LSTM, EST, voxel grid, graph-based). You may refer to Table 1 in ACGR [1] for a relevant comparison format.\n\n2. Although Appendix H provides latency analysis, it focuses only on comparisons within EVA itself. It would be more informative to include approximate inference time and computational complexity comparisons across different open-source representation methods (e.g., Matrix-LSTM, EST, voxel grid, graph-based). Since event representation is a fundamental technique, demonstrating its low-latency characteristics is particularly important.\n\n3. In future journal extensions, the authors could consider including more visualization results in the main text to better highlight the advantages of the proposed method. \n\n[1] Asynchronous Collaborative Graph Representation for Frames and Events, CVPR 2025.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "d6BLvF0iik", "reviewer": "Reviewer_A3qJ", "summary": "This paper addresses the sparsity and asynchrony of event camera data by proposing an Asynchronous-to-Synchronous (A2S) framework to better align event streams with deep learning methods. Inspired by language models, the authors treat event streams as ordered and incrementally evolving information, introducing Event Asynchronous Feature Learning (EVA), which leverages linear attention and self-supervised learning. Experimental results demonstrate that the proposed method achieves strong performance on both event-based recognition and inspection tasks.", "questions": "1. From Figure 4, it appears that MRP and NRP are not directly used in downstream tasks, but rather serve to guide the training of the event embedding module?\n\n2. In Figure 3C, the event embedding is transformed by the task head into a representation, which is then used to compute the loss together with features derived from MRP and NRP. How are MRP and NRP themselves trained in this setup?\n\n3. During the self-supervised learning process, how are MRP and NRP trained using handcrafted features? Are they trained separately in advance? Does this pretraining need to be done for each task or dataset individually? Why not directly use handcrafted features to train the event embedding instead?\n\n4. In Figure 6, what exactly are the learned representations, and what insights do they provide?\n\n5. Is the representation generalizable to all event-based downstream tasks, including optical flow estimation, semantic segmentation, etc., or is it restricted to specific tasks only?\n\n6. Compared with directly stacking event data into event images and processing them, what advantages does the proposed method offer? Does it provide benefits in terms of inference speed or computational cost (FLOPS)?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The motivation is clear. This paper specifically addresses the sparsity and asynchrony of event camera data and proposes a targeted solution. Treating continuous events as ordered natural language sequences and tokenizing them is a novel idea.\n\n2. The proposed method plays a positive role in bridging event camera data with deep learning approaches, effectively promoting progress in this research area.\n\n3. The writing is fluent overall, with a clear and coherent logical structure.", "weaknesses": "1. Insufficient experimental datasets. For the object recognition task, the authors only used relatively simple datasets such as N-Cars and DVS128-Gesture, without evaluating the method on more complex datasets like N-Caltech101 and N-ImageNet. This limitation may reduce the persuasiveness of the proposed approach.\n\n2. Low resolution of experimental datasets. The datasets used—N-Cars, DVS128-Gesture, and Gen1—all have relatively low resolutions, whereas modern event cameras can already reach 1280×720. Therefore, it is highly necessary to test the proposed method on high-resolution datasets. Moreover, during the A2S conversion, the resolution of the event camera can significantly influence the transformation speed. The authors should discuss how resolution affects the performance and efficiency of their method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)