FLAME: Fast Long-context Adaptive Memory for Event-based Vision
Authors: Biswadeep Chakraborty
Saibal Mukhopadhyay
Venue: NeurIPS 2025 poster ()
Keywords: state space models
event based vision
Adaptive Memory
sequence modeling
Primary area: applications
TL;DR: We introduce FLAMES, a novel spiking neural network architecture that effectively captures long-range temporal dependencies through Spike-Aware HiPPO and dendrite attention.

Abstract: We propose Fast Long-range Adaptive Memory for Event (FLAME), a novel scalable architecture that combines neuro-inspired feature extraction with robust structured sequence modeling
to efficiently process asynchronous and sparse event camera data. As a departure from conventional input encoding methods, FLAME presents Event Attention Layer, a novel feature extractor that leverages neuromorphic dynamics (Leaky Integrate-and-Fire (LIF)) to directly capture multi-timescale features from event streams. The feature extractor is integrates with a structured state-space model with a novel Event-Aware HiPPO (EA-HiPPO) mechanism that dynamically adapts memory retention based on inter-event intervals to understand relationship across varying temporal scales and event sequences. A Normal Plus Low Rank (NPLR) decomposition reduces the computational complexity of state update from $\mathcal{O}(N^2)$ to $\mathcal{O}(Nr)$, where $N$ represents the dimension of the core state vector and $r$ is the rank of a low-rank component (with $r \ll N$). FLAME demonstrates state-of-the-art accuracy for event-by-event processing on complex event camera datasets.

FLAME: Fast Long-context Adaptive Memory for Event-based Vision
Biswadeep Chakraborty Georgia Institute of Technology, Mercuria Energy Trading bchakrabortyy edd
Saibal Mukhopadhyay School of Electrical and Computer Engineering Georgia Institute of Technology
Abstract
We propose Fast Long-context Adaptive Memory for Event (FLAME), a novel scalable architecture that combines neuro-inspired feature extraction with robust structured sequence modeling to efficiently process asynchronous and sparse event camera data. As departure from conventional input encoding methods, FLAME presents Event Attention Layer, a novel feature extractor that leverages neuromor- phic dynamics (Leaky Integrate-and and-t-rr (LIF)) directly capture multi-timescal features from event streams. The feature extractor integrates with a structured state-space model with novel Event-Aware HiPPO (EA-HiPPO) mechanism that dynamically adapts memory retention based on inter-event intervals to understand relationship across varying temporal scales and event sequences. Normal Plus Low Rank (NPLR) decomposition reduces the computational complexity of state update from O(N2) to O(Nr). where N represents the dimension of the core state vector and is the rank of low-rank component (with r < N). FLAME demonstrates oftte-of-the-asse accuracy for event-by-event processing on complex event camera datasets.
Introduction
Neuromorphic cameras promise low operation power by generating asynchronous and sparse event data instead of dense pixel maps. Although spatial and temporal resolutions of event cameras are increasing 2 extracting features from long sequence of events remains challenging and computationally expensive 4 []. One class of event camera algorithms aggregate events over period of time into a volumetric representation 6 7 g 9. They show higher accuracy but incur higher computational cost and longer decision latency. Alternatively, wrent-by-event processing methods maintains spatiotemporal representation and updates that for every new event 10 4 [1]. They require less computation and lower latency, but often achieve lower accuracy complex datasets, simultaneously achieving low computational cost, event-by-event processing, and high accuracy for complex event camera data remains challenging.
Recent approaches for computationally efficient event camera processing primarily include methods based Graph Neural Networks (GNNs), Transformers, and Spiking Neural Networks (SNNs). While GNNs and Transformers can model complex inter-event relationships to achieve high accuracy, their core operations (e.g., graph convolution and attention mechanisms) typically process events in aggregated batches or windows 12
15 16 [7]. This operational paradigm fundamentally limits their ability perform continuous, low-latency event-by-event updates and often incurs high computational and memory demands, especially with large event volumes and long sequences [18 [9 20] Conversely, SNN-based methods inherently support event-by-event processing with promising computational efficiency. However, they often exhibit lower accuracy on complex tasks involving high spatial and temporal resolutions (i.e., longer event sequences). This performance gap for SNNs frequently stems from challenges in effectively training very deep architectures and the
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Inputs
Attertion
(b) Event Attention Layer
Branchse
Irput Events ||
Event Attention
Input
Raw Strae
tndradio veste
event time
Layer
Compute Time Difference
Spatial Pooling
Layer
Update Decay Matrt
Fy(CC))
Compune
EA-HiPPO
Convolution Layer
Memory Retention
1.0
EA HiPPO rotoils memory ofter spikes
ISPPO Eastanoo bacay Eeants)
EA-HIPPO
Updata
Convolution Layer
0.6
x(5x)))
K(w))
LayerNorm
0.4
Original
FFT based comolitiion
x(()
HSPO shown
0.2
seaddy decay
Compute 7/0
Readout Layer
0.0
Output Events
Time
||||||
(a) Overall FLAMES Model
(c) Event Aware HiPPO
(d) EA-HiPPO Convolution Layer
Figure 1: Block diagram of the proposed FLAME architecture. (a) Overall architecture, combining neuro-inspired feature extraction with efficient state-space modeling. (b) The Event Attention Layer (EAL) uses multi-branch Leaky Integrate-and and-Fire (LIF) dynamics to extract multi-timescale temporal features from raw event streams. (c) The core Event-Aware HiPPO (EA-HiPPO) dynamically modulates memory retention based event timing (Al), retaining context better than standard HiPPO after sparse events. (d) The EA-HiPPO Convolution Layer achieves efficiency via asynchronous updates, Planall-lls-owww-ee-
(NPLR) decomposition, and FFT-basee convolution.
inherent transient memory of basic spiking neuron models, which can impede the robust retention of contextual information extended event sequences.
State Space Models (SSMs) have emerged as an effective paradigm for modeling complex sequences They demonstrated good performance on event datasets but with event aggregation, and
21
22
24]. Advanced SSMs use HiPPO (High-Order Polynomial Projection Operator) framework (25 to compress and represent input's 's history through structured
hence, high computational cost [23
state-space dynamics. The SSMs built on HiPPO principles (e.g., S4 (21]) excel on dense, regularly sampled data such images. However, the HiPPO mechanism and subsequent SSM dynamics inherently assume continuous or eagulee inputs and require initial encoding layers to convert events into dense representations. Hence, they do not leverage the discrete, asynchronous, and sparse nature of event data for computational efficiency and event-by-eeeee by-eent processing.
This paper presents FLAME (Fast Long-context Adaptive Memory for Event-based Systems), novel, computationally efficient, and scalable SSM-based framework for event-by-event processing of lcage event-based vision data. The FLAAME architecture makes following key contributions.
Novel Event-Driven Input Encoding with Neuromorphic Dynamics: We propose the Event Attention Layer, which utilizes neuromorphic-inspree dynamics (such as Leaky Integrate-and and-FFre mechanisms) for direct, multi-timescale feature extraction from raw asynchronous event streams. This approach overcomes limitations of conventional input encoders for SSMs when applied to event data, by inherently processing information event- by-event and preserving crucial temporal precision and sparsity without requiring dense, fixed-size input patches or hand-crafted features.
Event-Aware HiPPO (EA-HiPPO) for Adaptive Memory: We present a novel adaptation of HiPPO principles where the state-space dynamics are made explicitly sensitive to the precise timing of discrete input events. Unlike standard HiPPO which assumes continuous inputs, EA-HiPPO dynamically modulates memory retention based inter-event intervals. This allows it to effectively capture diverse temporal patterns in varying timescales and maintain context within sparse and asynchronous event streams.
Computationally Efficient SSM Framework for Events: We apply Normal-Plus-LLow- Rank (NPLR) decomposition within the EA-HiPPO core, which reduces state update com-

---

plexity from O(N3) 21 to O The reduced computational cost makes FLAME
suitable for high-dimensional event streams.
Our theoretical analysis substantiates the FLAME framework, establishing bounds on memory capac- ity (26 27 versus computational cost for these event-driven SSM representations. Our experimental results on multiple event camera datasets, namely, HAR-DVS, Celex-HAR, 1-Caltech101, and CIFARIO-DVS, demonstrates that state-of-the-art accuracy with event-by-event processing but at a reduced computational cost (FLOPS/event) and processing latency. Crucially, on high-resolution data like CeleX-HAR, FLAME achieves comparable accuracy (s 72.2%) at R 90x lower com- pute (0.41 GFLOPs vs. 37.2 GFLOPs) compared to EventMamba, highlighting our efficiency focus.
Table 1: Qualitative Comparison of FLAME with prior methods
Dynamic Memory Retention
Direct Event Eyentt Event Long-Context Processing
Scalable
Low Computational Overbeud
Fully Asyechrommos Updates
Aduptability to Sparse Data
Model
Spocific Type
Memory
Graphh Yewval Natworks (GNN)
ADONN [S]
Asynchnonuus GNN
EvmICTS
Tracking GNN Classfictiin GNN
X
GeC
Transformtess
EventNet (AlaiiTFF)
Alert-driyzn TF Spiking TT
X Portial Partial
Partial
Portial
X Partial
Spiking Transformer
RVT
Recurrent Vision
Partial Partial
SpikecPP
Autoregresivve Syiking
Partial
BST
37
Spatio-leppune tempural TF
x
X
x
Spdking Nearal Networka (SNN)
DH-LIP
Dual threshold LIF
x
x
SNN
HOTS
Time Surface
x
(nnN-iis
SLAYER-SNY
SNN (generic)
Portial
LoCaSNN
100
jow-complexity SNN
x
Net
3H
LIF SNN w/ Attention
Portial
Partiol
State Space Models (SSM)
SpikingI.MU
88
Spiking LMU-TF (fgyrrid)
Partoll
Portial
BinaySADD 39
Binarized S4D (SSM)
Stractured SSM Hlybrid CNN
[0]
Portial
SSM-Exemt
Stractured
Partial
PRE Mamba
X
Mamba based SSM
Partiel
ALA (Chrs)
Event-Aware SSM
2 Related Works
Event Processing with Graph Neural Networks Graph Neural Networks (GNNs) model event
data by constructing graphs over event windows or directly exploiting event sparsity 29 28 While capable of capturing complex spatio-temporal relationships (c.g.. EventGTS (28), Get-Net (29).
GNNs often process aggregated event batches, limiting low-latency updates and incurring costs for graph management with large event volumes. Though some aim for asynchronous updates (c.g.. AEGNN [8), GNNs generally lack the inherent, scalable long-context memory mechanisms that FLAME's -SMebaaed architecture provides without explicit graph construction.
Event Processing with Transformers Transformers 41]. adapted for event data, range from operat- ing on event volumes (RVT (30), EST (32) to more event-driven variants like Arest-Transformer [[5] (partially asynchronous) and Spiking Transformers [14 31 (aiming for SNN efficiency). However, the core attention mechanism often remains computationally intensive for long event sequences or requires windowing strategics that can compromise fully asynchronous, low-latency updates 116 42 43). FLAME utilizes efficient SSM recurrence, bypassing the quadratic complexity of full attention for modeling long temporal contexts.
Spiking Neural Networks for Event Camera Processing Spiking Neural Networks (SNNs) natu- rally suit event cameras' asynchronous, sparse nature, promising efficiency 4. Standard training (SLAYER [35], surrogate gradients [4] 45 ) has advanced SNNs (e.g.. LoCoSNN (36)), which can process event-by-event. We choose LIF over simpler Integrate-and-Fire (IF) neurons because the leak is necessary to avoid unbounded build-up during long silent gaps and ensure training stability in deeper setups. However, traditional LIF neurons often struggle with extended temporal contexts due to rapid memory decay [46]. While enhancements like dendritic processing (DH-LIF (3 exist,
3

---

FLAME uniquely integrates principles from such mechanisms in its initial Event Attention Layer with its novel EA-HiPPO SSM core for robust long-context modeling, a common challenge for standalone SNNs.
State Space Models for Event Cameras State Space Models (SSMs) like S4 21 and Mamba [22]. built on the HiPPO framework [25], excel at lonu-eqqeence modeling but are designed for dense, regularly sampled data, making them sub-optimal for raw, sparse, asynchronous event streams
47 48 49]. Applying them to events often requires initial aggregation (e.g., into grids [23] or voxels
). potentially losing fine temporal details. While recent works (SpikingLMMU 38) BimaryS4D
principles, FLAME's Event-Aware HiPPO (EPP)) offers a distinct approach. It directly adapts core SSM state dynamics using measured inter-event time intervals, preserving HiPPO's long-context strengths while making memory evolution explicitly sensitive to event stream patterns.
399. SpikingSSMs [50, P. S- (SI)) explore bridging SSMs and
Adaptive Memory Mechanisms in Event-Driven Systems Adaptive memory is crucial for event- driven systems [45] [5 27). Approaches include Liquid Time-Constant (LTC) Networks [48] (adapting neuron time-constants) or STDP-like local learning [53]. FLAME's EA-HiPPO presents novel strategy: achieves adaptivity by globally and explicitly modulating a structured SSM's memory dynamics based on the timing of incoming events [26] (54). This direct modulation by event
statisties makes EA-HiPPO highly effective for efficient long-contex modeling in neuromorphic
vision [55], temporal reasoning [56], and resource-constraing edge applications [57 58 59 60).
3 Methods
FLAME introduces a novel neural network architecture engineered for the efficient processing of high- high-dimensional, asynchronous data from event cameras. It addresses the critical need for low computational overhead while effectively modeling complex spatio-temporal dynamics. The architecture (Figure [ strategically employs neuromorphiccinspired, event-driven computation for its initial feature extraction and then integrates this with a powerful SSM core for robust temporal modeling. This design philosophy allows FLAAME to directly process raw event data, harness its inherent sparsity, and overcome both the memory limitations of purely event-driven systems and the dense-input requirements of traditional SSMs. The data pipeline comprises: an Event Attention Layer for multi-timescale feature extraction from raw asynchronous events; subsequent Spatial Pooling Layer to enhance computational efficiency by reducing dimensionality: the core EA-HiPPO Convolution Layer for modeling long-ten sequential patterns through event-aware state-space dynamics; and finally, Layer Normalization and a Readout Layer for task-specific outputs.
Input Representation: FLAME directly processes input sequence of asynchronous events, the native output format of event cameras. Each event is typically a tuple (x, 1,,,)) denoting spatial coordinates, a precise timestamp, and polarity. This native, event-by-event processing is fundamental to FLAME's efficiency and its ability to preserve the rich temporal dynamics and sparsity of neuromorphic sensor data, computation is triggered only by new information. Our goal with "event-by-event is to ensure every update (feature extraction, pooling, and state updates) happens without batching or delay, not to imply that every stage is determined by single input spike alone.
Event Attention Layer The initial stage of FLAME, the Event Attention Layer, is designed to effectively capture the complex and varying temporal dynamics present in asynchronous event streams from high-dimensional event cameras. This layer is essential for transforming the raw, sparse event data into a richer feature representation while managing its inherent irregularity. Inspired by the multi-timescale filtering capabilities of dendritic branches in biological neurons, this layer is constructed using LIF neurons, each augmented with multiple conceptual branches (Figure [b)).. Each branch d is characterized by distinct learnable timing factor 7a, allowing it to act as temporal filter sensitive to a specific timescale. This is particularly important for event data where information can be encoded across wide spectrum of inter-event intervals.
The current ia(t) in conceptual branch d evolves according to:
dia(t)
1 ii()+)
+ Σ
way Ey JeNa
(1)
dt
Ta
4

---

or in its discrete-time form often used for simulation with a step stsmm (if applicable):
Ataio ia(t) (1-E
Atyim
ia(t Alsim) =
Σ JeNd
Waj
(2)
where (t) represents an input event from source j (binary, 1 if an event occurs at f. O otherwise), Waj are the learnable weights connecting input j branch d, and Na is the set of inputs to branch d. This multi-timescaa
filtering allows each neuron to effectively "attend" to features across diverse temporal windows within the sparse and irregular event stream, extracting meaningful patterns that might otherwise be lost.
These branch currents are then integrated at the neuron's soma T3 dV -V(( EBala(), where d
(II) is the membrane potential, Ts the somatic membrane time constant, and ga is the learnable coupling strength of branch d to the soma. This late fusion mechanism, using independent branches and separate coupling weights gd. is explicitly designed to minimize cross-talk between fast- and slow-timescale responses, the branches contribute separately to the final firing decision. An output occurs when (t) exceeds a threshold Vin. after which (t) is reset. The output of this layer is thus set of event trains, now enriched with multi-timescale temporal features, ready for further processing. The EAL acts as an effective denoising and temporal abstraction layer, suppressing isolated spikes while enhancing coherent spatio-temporal structures. This layer acts efficient front-end for raw event data, using LIF neuron dynamics for event generation which maintains sparsity.
Spatial Pooling Layer High-resolution event cameras (e.g., 1280 x 800) produce vast amounts of spatial data, posing computational challenges. To manage this efficiently, FLAME incorporates a Spatial Pooling Layer immediately after the Event Attention Layer The intuition is to reduce spatial dimensionality after initial rich feature extraction but before more complex temporal modeling, preserving crucial spatio-temporal details that might be lost if pooling raw, unprocessed input. Given an input event map I(x, t) from the Event Attention Layer, pooling (e.g., k x k max- pooling) produces [puoted (z', per event timestamp using a local max operation, without using any voxel grid or frame-level aggregation, fully preserving temporal resolution. Ablations show that applying pooling before the EAL caused consistent performance drop, justifying our chosen order. The output I ppooedd is typically flattened into (t) E {0, 11M time t, maintains precise event timing and significantly reduces the feature map size, critical for the computational tractability of subsequent layers. The output Epooted is typically flattened into (l (0, 1)" M for the EA-HiPPO layer.
max(z.))) ( (7(2,
t) This operation is performed
M for the EA-HiPPO layer.. This operation, performed at each event
EA-HiPPO Convolution Layer The core of FLAME's capability to efficiently model long temporal contexts from event data resides in the EA-HiPPO Convolution Layer. This layer adapts State Space Model principles, particularly the HiPPO framework (25). to the sparse, event-driven nature of the processed event vector Ef) While termed "convolution" due its conceptual link to efficient parallel training modes of modern SSMs (e.g., S4), its inference is inherently recurrent and event-driven, aligning with low-compute goals.
state vector x(t) E N captures historical context, evolving as xx()) = At(xx(()) BE Bat (t)
The novelty here the time-varying adaptive state transition matrix As(() = As(A()= = A F(AI). Here, A is a base HiPPO matrix, and F (t)) is a dynamic decay modulation matrix (with elements (At) e e- At and learnable 0z > 0) that adapts to the inter-event interval At. This Event- Aware HiPPO (EA-HiPPO) design allows memory dynamics to respond to event sparsity: frequent events (At a 0) lead to Ag v A (preserving memory), while sparse events (large At) allow faster decay of specific memory elements.
The matrix is initialized using the HiPPO-LegS formulation, which guarantees that all its eigenval- ues satisfy Re(Ax(A A)) < 0 (a Hurwitz matrix), ensuring the continuous-time system is asymptotically stable. This stability is preserved in the event-adaptive variant A (A)) because the decay modulation matrix F (At) (0, 1] does not flip the sign of the eigenvalues.
5

---

This dynamic adaptation is crucial for asynchronous event streams. For event-driven discrete-timg updates (interval At,, input Eevente.)) ):
x(1==1) (s(A)))))*x(1)))) + (As()
I)
(3)
The matrix exponential is efficiently approximated using second-order Taylor expansion. This, along with Normal- Dormal-Plus-Low-RRnnk Low-Rank (NPLR) decomposition of A (reducing complexity to O(Nr)) ensures low computational cost. Specifically, we use N 64 and fix the rank r in all experiments. The output state x(t) thus encodes long-term sequential patterns informed by precise event timings.
Layer Normalization and Readout Layer Following the EA- Convolution Layer, Layer Normalization is applied to the state representation x(t) to stabilize learning from sparse and dynamic event-driven activations. The final Readout Layer then maps these learned temporal representations to task-fpecific predictions. It typically aggregates information from the state sequence x(t1), ..... .(tt) (e.g., via temporal averaging of the last K states to get Xpooled) and passes this through a linear layer with an appropriate output activation (e.g., softmax for classification, yredd Softmax W out ppoted bout)). ). This provides efficient mechanism derive final outputs.
4 Theoretical Analysis
This section establishes theoretical guarantees for FLAME, our adaptive state-space framework. We focus on three key aspects: computational efficiency of state updates, the ability to model extended temporal contexts from event data, and overall system stability. Formal bounds are presented to analyze how FLAME's components ensure robust and efficient processing. Detailed proofs for all theorems and lemmas are provided in the Suppl. Material.
4.1 Computational Complexity: Enabling Efficient Event Processing
Lemma (Computational Efficiency of Adaptive State Updates). The state update for the EA-HiPPO
dynamics, x(t) As(At)x(t) + BE (t) (as defined in Sec. 3 where Estae()) is the flattened
input event vector), with x(1) R. and (At)) = A F F(Dt), has complexity of O(N') per input event if the base HiPPO matrix A E B.NN is dense. With Normal-Plus-Loww Plus-Low-Rank (NPLR) decomposition applied to A, this complexity reduces to ((Nr) per input event, where r < N is the rank of the low-rank component, provided F(At) allows for efficient clement-wise operations.
This lemma highlights cornerstone of FLAME's efficiency: the NPLR decomposition drastically reduces the computational cost each state update from quadratic to near-lineal (O(Nr) in the state size N. This reduction is essential for real-time processing of potentially high-rate event data, as the core operations involving the adaptive state matrix As (At) remain computationally tractable.
4.2 Modeling Extended Temporal Contexts in Event Streams
Theorem 1 (Preservation of Information over Extended Durations). Let E R evolve according the EA- EA-HiPPO dynamics:
x(t)= As(At)x(t) (ia
a base HiPPO matrix with eigenvalues (A satisfy- ing Re( Re)) O, F(AI) is the adaptive decay matrix (Fg(At)) = -a, inputs satisfy If all eigenvalues du (As((tt)) of the adaptive state matrix As(At) have negative real parts for relevant inter-event intervals At, then the state norm is bounded:
where As(At) A F(At),
(E | & E... and x(0)
B
[x(t)| S -eest [xoll
(1- (1-e-aesrt)
Ceff
where defs= = ming (Re(As(As (at))) (As(tA)))))) 0 the memory retention rate, that adapts with At.
Theorem 1 demonstrates FLAME's capacity to maintain information over long durations and un- derstand temporally distant relationships within event streams. The EA-HiPPO dynamics allow the effective memory retention rate, Oeff, to be small when necessary, preserving long-term context. Crucially, the adaptive nature of A (A), modulated by F(A() based on input event sparsity (inter- event interval At), allows this retention to be dynamically adjusted, aligning with the principles described in Sec. 3

---

4.3 Numerical Stability of Event-Driven State Updates
Lemma 2 (Stability of Taylor Expansion for State Update). Approximating the matrix exponential ( using an n-th order truncated Taylor series (Sec. B the single-step approximation error En is bounded by:
(As()t) clAAs(A))AN))
(n+ 1)!
This bound (Lemma 2) justifies using a low-order Taylor expansion (e.g., n 2, as in Sec. 3 for the matrix exponential. This balances computational efficiency with numerical stability, ensuring reliable updates even with varying inter-event intervals At.
Theorem 2 (Global System Stability). The state trajectory x(t) of FLAME, governed by x(t) As( (st)))((() faa remains bounded (i.e., ||x(t)] C for some constant C > 0, Vt 2 0) if:
1. Bounded Inputs: Input event streams ensure (t) Ess, 2
2. Uniform Hurwitz Stability of (tt)):: For encountered inter-event intervals At, As(At) is uniformly Hurwitz (all eigenvalues Au Ae(As(At) satisfy Re(Nx s -E for some fixed 0).
3. Lyapunov Condition:
For each relevant As(At). E positive definite P(At) s.t. As +(A)))+))))))))))))))))))
= Q(4t) for some positive definite Q
Theorem 2 ensures the overall stability of FLAME. It guarantees that the internal state representation remains bell-be and does not diverge, even when processing extended and sparse event streams, provided the adaptive state matrix s (At) maintains stability across the typical dynamics of inter- event intervals.
Our theoretical analysis highlights how FLAME engineered for efficient weeee-based processing. The NPLR decomposition (Lemma 1) is fundamental to achieving low per-event update costs (O(Nr)), enabling the model to handle complex memory representations from event data without prohibitive computation. Concurrently, the EA-HiPPO mechanism's 's adaptive state matrix As (At) dynamically modulates the memory retention rate aeʃʃ (Theorem 1) response to input event sparsity, using the learnable decay matrix F(At). This creates intrinsic balance between preserving information over extended temporal contexts and adapting to new, incoming events. Coupled with guarantees of numerical and global system stability (Lemma> 2, Theorem 2), these poorrtically-groundee design choices make FLAME particularly suitable for robust and soource-aware processing of nigh-dimensional event streams.
Discussion:
Balancing Efficiency and Memory for Event Data:
Experiments and Results
We conduct empirical evaluation of FLAME to demonstrate its effectiveness and efficiency in pro- cessing asynchronous, event-driven data from event cameras across various challenging benchmarks. Our experiments are designed to assess: (I) performance on demanding event-based vision datasets, including high-resolution and complex activity scenarios; (2) computational efficiency in terms of FLOPs, parameters, and inference latency, including profiling on diverse hardware backends; and (3) the specific contributions of key architectural components through targeted ablation studies.
5.1 Experimental Setup
Datasets: We evaluate FLAME on a diverse set of public benchmarks to assess its capabilities in
event-based vision. For event-based vision. these include DVS Gesture 77] (gesture recognition). HAR-
DVS [6]] (human activity), Celex-HAR HAR [2] (high-resolution human activity), CIFARIO-DVS [62 (neuromorphic image classification), and N-Caltech101
[63 (neuromorphic object recognition).
Additional evaluations cover weent-based speech datasets, namely Spiking Heidelberg Digits (SHD)
and Spiking Speech Commands (SSC) [64], and Sequential CIFAR-10/100 65] for sequential image
classification, with further details provided in Appendix B
Across all datasets, FLAME processes inputs event- by-een,, dynamically updating its hidden state with each incoming event to preserve high temporal resolution. Comprehensive details on all dataset characteristics and specific task setups available in Appendix B.I
7

---

-00
FLANES Awrt-kama
ON
6.3
Bact
1014
No Denditt
10.000
300 serrO
CHH Nodels
Sowfaar
Winitoraed
dreer
Hunboren
Thansfemer Necebs
Models (Dars)
FLaree
FLAMES Teyy
Chi Modals
MTT Modals
Modilc
5GM Modal:
SP18 Models
1M4
Rwey Models
SPLB
5SN shotets
PLOPs (log)
FLOPs (D) Blog)
10
10
(a)
(b)
(c)
Figure 2: Accuracy versus GFLOPs across various event-based vision datasets, comparing FLAME variants with other State-of-the-Art (SOTA) models. (a) Performance on DVSGesture 28, including ablation studies for FLAME demonstrating the impact of removing the Event Attention Layer (No Dendrite) or replacing EA-HiPPO with standard LIF neurons (No SA-HiPPO), (b)
Performance on the high-resolution CeleX-HAR dataset, showcasing FLAAM''s efficiency at scale. (c)
Performance on HAR-DVS, highlighting the competitive accuracy of FLAMM variants. Note: FLAME variants consistently occupy favorable positions, indicating superior accuracy for their computational budget.
Table 2: Performance and Efficiency Comparison of FLAME-Normal against State-of-the-Ar (SOTA)
models across various event- base datasets. Dataset sizes are approximate. FLOPs GigaFLOPs.
Datasel
Resolution
Size (Samples; Classes)
SOTA Model (Acc. %)
SOTA (L)Ps(G)
FLAME- Normal (Acc. %)
FLAME-Normal FLOPS (G)
IO-DS
128 128
110k Events/ing: 10 Cls
Spike-GGG1
8.9
80.5%
0:43
(55.11%)
1-
Orig: 302 245
-8.7k Samples; 101 Cls
BPV++ (89779)
70.5%
1.34
(varies) 128 128
DVS128 Gesture
-1 1.33 Samplex; 11 Cl
EveniMamba (99.2%)
0.219
96.5%
0.43
HAR-DVS 3I
316 260
1. Samples; Cls
EXACT (90.10%)
1.3
88.29%
0:11
Colx- HAR H
1280 x 800
-1 125k Samples; Clb
EVMambu (72.3%)
37.2
72.2%
0.41
We evaluate three primary variants of FLAME-Tiny Small, and Normal-to demonstrate scalability and the trade-offs between performance and computational complexity. Their architectural details (e.g.. number of distinct timing factors in the Event Attention Layer, filter counts, readout layer neurons) are provided in Suppl. Sec. C All variants employ the EA-HiPPO mechanism.
FLAME Model Variants:
Evaluation Metrics and Implementation: We report accuracy, GFLOPs (Giga Floating Point Operations), number of parameters, and inference latency (ms). FLAME models are implemented in PyTorch. Main training and inference for event datasets are conducted on NVIDIA A100 GPUs. Further details on hyperparameters and training procedures are in Appendix B
5.2 Performance on Event-Based Vision Benchmarks
FLAME demonstrates strong performance across multiple event-based vision datasets, often achieving a superior accuracy-efficiency trade-off. This highlights its suitability for processing high-dimensional event camera data with low compute requirements.
DVS Gesture: As shown in Figure 2 (a., FLAME-Normal obtains 96.5%, while FLAME-Sman (93.7%) and FLAME-Tiny (89. (89.2%) offer graceful performance degradation at lower compultation. These results compare favorably against models like EventMamba [40] and PointNet++ [66] but at much fewer FLOPS. Detailed comparisons in Suppl. Table
Contrast with EventMamba: This comparison highlights the models' architectural focus. While EventMamba achieves slightly higher accuracy on the low-resolution DVS Gesture dataset (99.2% vs. FLAME's 96.5%) at comparable FL.OPs, FLAME demonstrates its architectural advantage in processing high-resolution, sparse data on CeleX-HAR, Here, FLAME maintains comparable
8

---

Perfermenee: 72.35
EVMambee
Maraaa
0.72
Vitamt-8 & 0/9
RaMES Sarel
7DM
0.70
VistonPamme-S
STVH
0.78
V-Swin Trans
FLAMES Small
[0.3%
0.68
Accuracy (%)
Accuracy
Slowfast
VideoMamto
0.66
0.46
www
0.64
Reshertio
CNN Models
0.64
Models
Cap
FLAMES (53.2%)
RWKV Models
SSM Models
SPLR Models
(Ours)
Transforner Nodels
Space Modala
167
10P
10
10
10
1G
Parameters (M)
Latency (ms) (log)
(a)
(b)
Figure 3: Efficiency analysis on the CeleX-HAR dataset, measured on an NVIDIA A100 GPU. FLAME variants demonstrate superior trade-off compared to SOTA models. (a) Accuracy versus Parameters (M): FLAME achieves competitive accuracy with significantly lower parameter counts than many high-performance models. (b) Accuracy versus Inference Latency (ms) (log scale): FLAME models exhibit substantially lower latency, confirming the efficiency of the asynchronous, event-by-event design for real-time applications.
accuracy (72.2%) while operating at & 90x lower computational cost (0.41 GFLOPs vs. 37.2 GFLOPs), showcasing superior efficiency and scalability. We detail this trade-off further in Appendix B.4.
HAR-DVS: On this dataset for human activity recognition from event streams (Figure (c)), FLAME shows strong performance, outperforming several state-of-the-art DNNs, with lower computational cost.
CeIex-HAR (Scaling to HD Event Streams): Figure ( illustrates FLAME's scalability to high-resolution (1280 x 800) event streams. PLAME-Norma. surpasses the accuracy of models like TSM 67) and VisionMamba-S [68] while operating with significantly lower GFLOPs. This highlights EA-HiPPO's ' effectiveness in managing spatio-temporal complexity in high-dimensional event data efficiently.
Other Datasets: FLAME also shows strong performance on other event datasets like CIFARIO-DVS. NCaltech SHD and SSC event-based speech datasets as shown in Suppl. Table
For completeness, we note that comparisons with voxel-based models like EventMamba are omitted for datasets such as SHD and SSC because those models rely on frame-wise or voxel-based recon- struction, which is not directly applicable to classification tasks where ground-truie labels are defined the sequence level.
5.3 Efficiency Analysis
Figure provides a comparative visualization of accuracy versus GFLOPs across DVSGesture128, Celex-HAR, and HAR-DVS. FLAME variants consistently occupy favorable positions, indicating superior accuracy for their computational budget compared to many established models. For instance, on DVSGesture FLAME-Normal achieves high accuracy with significantly fewer parameters and GFLOPs than several competing methods (further details in Appendix B). The methodology for estimating FLOPs, particularly for event-driven components, is detailed in Appendix|C.. C.7.
FLOPs and Parameters:
Inference Latency: We measured inference latency on NVIDIA A100 GPU (40GB VRAM).
As shown in Figure 3 for the challenging Celex-HAR dataset, FLAME models exhibit significantly lower latency, crucial for real-time event processing. FLAME- Tiny achieves the lowest latency

---

at 0.162 ms. FLAME-Smin (0.582 ms) and PLAME-Norman (1.867 ms) also maintain strong lacercy-accuracy profile compared to models like TimeStormer (255.425 ms) and even optimized alternatives like VideoMamba-S (19.707 ms). Detailed latency figures are in Suppl. Table g To further assess practical deployability on diverse hardware, we profiled FLAME-Tinn across various platforms, including CPUs and other GPU tiers. These results, detailed in Appendix B.6 (Table 10). demonstrate consistent accuracy and highlight FLAME's adaptability.
Ablation Studies: To validate the contributions of FLAME's key architectural components to its overall performance and efficiency, we conducted ablation studies, primarily on the DVS Gesture 128 dataset (see Figure dashed lines and Suppl. Table for full details).
Impact of Event Attention Layer: Removing the multi-timessee Event Attention Layer resulted in a noticeable accuracy drop across all FLAME variants (e.g., FLAME-Normal accuracy decreased from 96.5% 95. 95.2%). This highlights the importance of this specialized initial processing stage for effectively capturing diverse temporal features present in raw event streams.
processing mechanism within the
Impact of EA-HiPPO: Replacing the EA-HiPPO mechanism with standard LIF neurons keeping other architectural elements comparable for basic event processing) led to significant performance degradation (e.g.. FLAME-Normal accuracy further reduced to 90.4%). This underscores the critical role EAAHHiPOO's ss structured, adaptive memory in retaining long-range temporal context from event data, capability that simpler neuron models inherently lack for complex sequences.
These ablations confirm that both the Event Attention Layer and the EA-HiPPO mechanism are integral to FLAAME's strong performance and its efficiency in processing event-based data.
6 Conclusion
In this paper, we introduced FLAME, novel architecture engineered for the efficient processing of large-scale, asynchronous event-based data. By integrating an event-driven, neuromorphic-inspire front-end with powerful state-space memory core, FLAME achieves compelling performance. Its Event-Aware HiPPO (EA-HiPPO) mechanism dynamically adapts memory based on inter-event intervals, enabling robust modeling of complex temporal patterns and the understanding of temporally distant relationships in sparse data streams. Our comprehensive evaluations demonstrate FLAME's significant advantages in accuracy, computational efficiency (low FL.OP//ltency)), and scalability across diverse benchmarks, including challenging wvent-based vision tasks and temporal reasoning. establishing it as a capable framework for processing real-world event data on conventional hardware.
Current I. Limitations and Future Directions. Despite its efficiency, the current FLAME framework has practical limitations. While it achieves high throughput modern GPUs, it does not yet meet real-time constraints on CPU-only systems and is designed for static offline training, currently lacking mechanisms for on on-device continual learning. Furthermore, in high-noise environments may be sensitive to sensor noise, a known trade-off mitigated but not eliminated by the EAL filtering-
event-driven model, its performance
However, while FLAME incorporates event-driven principles, its core EA-HiPPO mechanism, as sophisticated state-space model, is not a purely spiking neuromorphic algorithm in its current formulation. This means the full architecture not directly translatable for end-to-end native execution on existing neuromorphic hardware platforms. A key direction for future work is therefore to adapt and extend the principles of FLAME, particularly its event-aware state dynamics, to enable efficient mapping onto such specialized hardware. Our goal is to leverage these advancements for ultra-low- low--err power, large-scale, and efficient event- based vision at the edge, fully realizing the potential of neuromorphic computing for these demanding applications.
Acknowledgements
This work is supported by the Defense Advanced Research Projects Agency (DARPA) under Grant Numbers N66001 2324003. The views and conclusions contained in this document are those of the authors and should not interpreted as representing the official policies, either expressed or implied, of the Department of Defence, DARPA, or the U.S. Government.
V660012320003.
10

---

Broader Impacts
The primary societal impact of FLAME is derived from its architectural focus on efficiency and real-time processing for event-based vision. The core positive impact is enabling the deployment of sophisticated vision systems in resource--onstrained environments. By achieving competitive accuracy high-resouttion data substantially lower computational costs and ultra-low latency, this work significantly lowers the hardware and power barrier for applications in areas such as remote environmental monitoring, advanced robotics, and autonomous navigation at the edge.
As foundational architecture research that utilizes only established, publicly available benchmark datasets, this work introduces unique negative societal risks beyond those generally associated with technological progress in machine perception. We acknowledge that enhanced real-time sensing capability could be potentially applied to advanced surveillance systems, but our focus remains on promoting the responsible use of this technology, specifically for resource- resource-efficien and beneficial applications.
References
[1] Shoushun Chen and Menghan Guo. Live demonstration: Celex-v: a Im pixel multi-mode event- based sensor. In 2019 IEFF/CVE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 1682-1683. IEEE, 2019.
[2] Xiao Wang, Shiao Wang. Chuanming Tang, Lin Zhu, Bo Jiang. Yonghong Tian, and Jin Tang. Event stream-based visual object tracking: A high-resolution benchmark dataset and novel baseline. arXiv preprint arXiv:2408. 09764 pages 19248-19257 2024.
[3] Cedric Scheerlinck, Henri Rebecq. Timo Stoffregen, Nick Barnes, Robert Mahony, and Davide Scaramuzza. CED: Color Event Camera Dataset. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 591-600, June 2019.
[4] Guillermo Gallego, Tobi Delbruck, Garrick Orchard, Chiara Bartolozzi, Ben Taba, Andrea Censi, Stefan Leutenegger, Andrew J Davison, Jörg Conradt, Kostas Danilidiis, and Davide Scaramuzza. Event-based vision: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(1):154-180 2020.
[5] Catherine D Schuman, Shruti R Kulkarni, Maryam Parsa, Parker Mitchell, Prasanna Date, and Bill Kay. Opportunities for neuromorphic computing algorithms and applications. Nature Computational Science, 2(1):10-19, 2022.
[6] Alex Zihao Zhu, Lixu Yuan, Kendall Chaney, and Kostas Daniilidis. Unsupervised event-based learning of optical flow, depth, and egomotion. Proceedings of the IEEE Conference on Computer Vision and Patlern Recognition Workshops (CVPRW), pages 1-10. IEEE, 2018.
[7] Arnon Amir, Brian Taba, David Berg, Timothy Melano, Jeffrey McKinstry, Carmelo Di Nolfo, Tapan Nayak, Alexander Andreopoulos, Guillaume Garreau, Marcela Mendoza, al. A low power, fully event-based gesture recognition system. In Proceedings of the IEEE Conference on Computer Vision and Pottern Recognition, pages 7243-7252. IEEE, 2017.
[8] Xuan Peng, Michael Streeke, and Guido Reina. Event-based, direct camera tracking and mapping in real-time. In 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pages 326-335. IEEE, 2020.
[9] Henri Rebecq, Elias Mueggler, Guillermo Gallego, and Davide Scaramuzza. EMVS: Event- based multi-view stereo. In European Conference on Computer Vision (ECCV), pages 3-20. Springer, 2016.
[10] Valentina Vasco, Arren Glover, and Chiara Bartolozzi. Event-based harris corner detection for neuromorphic vision sensors. In 2016 IEFF/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5101-5107. IEEE, 2016.

---

[11] Ryad Benosman, Charles Clereq, Xavier Lagorce, Sio-Hoi Ieng, and Chiara Bartolozzi. Event-based visual flow. IEEE Transactions on Neural Networks and Learning Systems, 25(2):407-417, 2014.
[12] Yin Bi, Aaron Chadha, Alhabib Abbas, Eirina Bourtsoulatze, and Yiannis Andreopoulos. Graph-based spatio-temporal feature learning for neuromorphic vision sensing. IEEE Transac- tions on Image Processing, 29:9084-9099 -998,, 2020.
[13] Yongjian Deng, Hao Chen, Hai Liu, and Youfu Li. A voxel graph enn for object classification with event cameras. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pages 1172-11811 1181, 2022.
[14] Man Yao, Jiakui Hu, Zhaokun Zhou, Li Yuan, Yonghong Tian, Bo Xu, and Guoqi Li. Spike- driven transformer. Advances in neural information processing systems, 36, 2024.
[15] Carmen Martin Turrero, Maxence Bouvier, Manuel Breitenstein, Pietro Zamuttigh, and Vincent Parret. crotttrranformmr:: Bridging asynchronous and synchronous machine learning for real-time event-based spatio-temporal data. arXiv preprint arXiv:2402.01393, 2024.
[16] Biswadeep Chakraborty and Saibal Mukhopadhyay. Exploiting heterogeneity in timescales for sparse recurrent spiking neural networks for energy-efficient edge computing. SRC TechCON 2024, 2024.
[17] Mathias Gehrig and Davide Scaramuzza. Recurrent vision transformers for object detection with event cameras. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 13884-1 1044--18992 13893, 2023.
[18] Simon Schaefer, Daniel Gehrig, and Davide Scaramuzza. Aegnn: Asynchronous event-based graph neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12371-123811 2022.
[19] Nico Messikommer, Daniel Gehrig, Antonio Loquercio, and Davide Scaramuzza. Event-based asynchronous sparse convolutional networks, In European Conference on Computer Vision, pages 415-431. Springer, 2020.
[20] Yi Tay, Mostafa Dehghani, Dara Bahri, and Donald Metzler. Efficient transformers: A survey. ACM Computing Surveys (CSUR), 1-(6):1-28, 2022.
[21] Albert Gu, Karan Goel, and Christopher Re. Efficiently modeling long sequences with structured state spaces. In International Conference on Learning Representations, 2022.
[22] Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752, 2023.
[23] Nikola Zubic, Dragana Zovkic, Anas Sadel, and Davide Scaramuzza. State Space Models for Event Cameras. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2024.
[24] Ciyu Ruan, Ruishan Guo, Zihang Gong, Jingao Xu, Wenhan Yang, and Xinlei Chen. PRE- Mamba: A 4d state space model for urXiv:2405.05307 2024.
event camera deraining. arXiv preprint
[25] Albert Gu, Tri Dao, Stefano Ermon, Atri Rudra, and Christopher Ré. Hippo: Recurrent memory with optimal polynomial projections. Advances in neural information processing systems, 13:147441487, 2020.
[26] Wenjie Wei, Malu Zhang, Jilin Zhang, Ammar Belatreche, Jibin Wu, Zijing Xu, Xuerui Qiu, Hong Chen, Yang Yang, and Haizhou Li. Event-driven learning for spiking neural networks. arXiv preprint 2rXiv:2403.00270 2024.
[27] Biswadeep Chakraborty and Saibal Mukhopadhyay. Heterogeneous recurrent spiking neural network for spatio-temporal classification. arXiv preprint arXiv:2211.04290 17:994517, 2022.
12

---

[28] Lei Shi, Yezhou Wang, and Shang Xu. Eventgts: Event-based graph tracking system with spatio-temporal pyramid attention. In Proceedings of the IEEE/CVF International Conference Computer Vision (ICCV), pages 12566-12576, 2023.
[29] Zhao Bi, Aayush Chadha, Ahmed Abbas, Eirina Bourtsoulatze, and Yiannis Andreopoulos. GEC-Net: Graph-embedded event camera classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 643-653, June 2020.
[30] Daniel Gehrig, Mathias Gehrig, Javier Hidalgo-Carri, and Davide Scaramuzza. Video to events: Recycling video datasets for event cameras. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3986-3595, 2020.
[31] Yiming Zhu, Bohan Yin, Jianing Chen, Zhaofei Liu, Deli Zhao, Boxin Li, Tiejun Mei, and Feng Liu. SpikeGPT: Generative pre-trained spiking neural network for event-based vision. In Advances in Neural Information Processing Systems (NeurIPS), volume 36, pages 34253-
34267, 2023.
[32] Daniel Gehrig, Antonio Loquercio. Konstantinos G Derpanis, and Davide Scaramuzza. End- to-end learning of representations for asynchronous event-based data. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5633-5643, 2019.
[33] Hanle Zheng, Zhong Zheng, Rui Hu, Bo Xiao, Yujie Wu, Fangwen Yu, Xue Liu, Guoqi Li, and Lei Deng. Temporal dendritic heterogeneity incorporated with spiking neural networks for learning multi-timescale dynamics. Nature Communicationn, 15(1):277, 2024.
[34] Xavier Lagorce, Garrick Orchard, Francesco Galluppi, Bertram E Shi, and Ryad B Benosman. Hots: a hierarchy of event-based time-surfaces for pattern recognition. IEEE transactions on pattern analysis and machine intelligence, 396))1344-1335, 2016.
[35] Sumit Shrestha and Garrick Orchard. Slayer: Spike layer error reassignment in time. Advances in Neural Information Processing Systems (NeurIPS), 31, 2018
[36] Federico Paredes- Parldes-Vallés, Julien Dupeyroux, and Guido C H E de Croon. LoCoSNN: A LOw- COmplexity spiking neural network for event-based optical flow. Neural Computing and Applications, 2024.
[37] Zhenzhi Wu, Hehui Zhang, Yihan Lin, Guoqi Li, Meng Wang, and Ye Tang. Liaf-net: Leaky integrate and analog fire network for lightweight and efficient spatiotemporal information processing. IEEE Transactions Neural Networks and Learning Systems, 33(1 (3(11))6229-6262, 2021.
[38] Zeyu Liu, Gourav Datta, Anni Li, and Peter Anthony Beerel. Lmuformer: Low complexity yet powerful spiking model with legendre memory units. arXiv preprint arXiv:2402.04882 2024.
[39] MI Stan and O Rhodes. Learning long sequences in spiking neural networks using state-space models. Scientific Reports, 14(1):21957, 2024.
[40] Hongwei Ren, Yue Zhou, Jiadong Zhu, Haotian Fu, Yulong Huang, Xiaopeng Lin, Yue- tong Fang, Fei Ma, Hao Yu, and Bojun Cheng. Rethinking efficient and effective point- based networks for event camera classification and regression: Eventmamba. arXiv preprint arXiv:2405.061 006.26666..666,.. 2024.
[41] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, tukasz Kaiser, and Illia Polosukhin. Attention all you need. In Advances in Neural Information Processing Systems (NewrIPS), volume 30, 2017.
[42] Biswadeep Chakraborty, Uday Kamal, Xueyuan She, Saurabh Dash, and Saibal Mukhopadhyay. Brain-inspired spatiotemporal processing algorithms for efficient wvent-based perception. In 2023 Design, Automation & Test in Europe Conference & Exhibition (DATE), pages 1-6. IEEF,
2023.
13

---

[43] Hemant Kumawat, Biswadeep Chakraborty, and Saibal Mukhopadhyay. Robokoop: Efficient control conditioned representations from visual imput in robotics using koopman operator. CoRI. 2024, 2024.
[44] Emre O Neftei, Hesham Mostafa, and Friedemann Zenke. Surrogate gradient learning in spiking neural networks. IEEE Signal Processing Magazine, 36(6):61-63, 2019.
[45] Guillaume Bellec, Darjan Salaj, Anand Subramoney, Robert Legenstein, and Wolfgang Maass. Long short-term memory and learning-to-les in networks of spiking neurons. Advances in neural information processing systems, 31, 2018.
[46] Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, and Luping Shi. Spatio-temporal backpropagation for training high-performance spiking neural networks. Frontiers in neuroscience, 12:331, 2018.
[47] Albert Gu, Isys Johnson, Karan Goel, Khaled Saab, Tri Dao, Atri Rudra, and Christopher Ré. Combining recurrent, convolutional, and continuous time models with linear state space layers. Advances in neural information processing systems, 144572-585, 2021.
[48] Ramin Hasani, Mathias Lechner, Yulia Yildiz, Radu Grosu, and Daniela Rus. Liquid time- constant networks. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAT), 2021.
[49] Biswadeep Chakraborty and Saibal Mukhopadhyay. Topological representations of heteroge- neous learning dynamics of recurrent spiking neural networks. IJCNN 2024, 2024.
[50] Shuaijie Shen, Chao Wang. Renzhuo Huang, Yan Zhong, Qinghai Guo, Zhichao Lu, Jianguo Zhang, and Luziwei Leng. Spikingssms: Learning long sequences with sparse and parallel spiking state space models. Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 00380-20388 2025.
[51] Malyaban Bal and Abhronil Sengupta. P-spikessm: Harnessing probabillistic spiking state space models for long-range dependency tasks. arXiv preprint arXiv:2406.02922; 2024.
[52] Friedemann Zenke and Emre O Neftei. Brain-inspired learning on neuromorphic substrates. Proceedings of the IEEE, 109(5):935-950, 2021.
[53] Biswadeep Chakraborty and Saibal Mukhopadhyay. Characterization of generalizability of spike timing dependent plasticity trained spiking neural networks. Frontiers in Neuroscience, (arXiv preprint 1.XXi:2205.116::1 2021.
[54] Albert Gu, Isys Johnson, Aman Timalsina, Atri Rudra, and Christopher Ré. How to train your hippo: State space models with generalized orthogonal basis projections. arXiv preprint arXiv:2206. /2037, 2022.
[55] Daniel Gehrig and Davide Scaramuzza. Low-latency automotive vision with event cameras. 1043, 2024.
Nature,
[56] Mingqing Xiao, Yixin Zhu, Di He, and Zhouchen Lin. Temporal spiking neural networks with synaptic delay for graph reasoning. In Forty-first International Conference on Machine Learning, 2024.
[57] Biswadeep Chakraborty, Beomseok Kang, Harshit Kumar, and Saibal Mukhopadhyay. Sparse spiking neural network: Exploiting heterogeneity in timescales for pruming recurrent snn. In The Twelfth International Conference on Learning Representations (ICLR). arXiv preprint arXiv:2403.03440, 2024.
[58] Biswadeep Chakraborty and Saibal Mukhopadhyay. Brain-inspired spiking neural network for online unsupervised time series prediction. In 2023 International Joint Conference on Neural Networks (IJCNN), pages 1-8. IEEF, 2023.
[59] Hemant Kumawat, Biswadeep Chakraborty, and Saibal Mukhopadhyay. Stage net: Spatio- temporal attention-base based graph encoding for learning multi-agent interactions the presence of hidden agents. 2024.
14

---

[60] Biswadeep Chakraborty, Harshit Kumar, and Saibal Mukhopadhyay. A dynamical systems- inspired pruning strategy for addressing oversmoothing in graph neural networks. arXiv preprint arXiv:2412.07243, 2024.
[61] Xiao Wang. Zongzhen Wu, Bo Jiang, Zhimin Bao, Lin Zhu, Guoqi Li, Yaowei Wang, and Yonghong Tian. Hardvs: Revisiting human activity recognition with dynamic vision sensors. In Proceedings of the AAAI Conference Artificial Intelligence, volume 38, pages 5615-5623, 2024.
[62] Hongmin Li, Hanchao Liu, Xiangyang Ji, Guoqi Li, and Luping Shi. Cifar 00-dss: event- stream dataset for object classification.. Frontiers in neuroscience, 11:309, 2017.
[63] Garrick Orchard, Ajinkya Jayawant, Gregory K Cohen, and Nitish Thakor. Converting static image datasets spiking neuromorphic datasets using saccades. Frontiers in neuroscience, 9:437, 2015.
[64] Benjamin Cramer, Yannik Stradmann, Johannes Schemmel, and Friedemann Zenke. The heidelberg spiking data sets for the systematic evaluation of spiking neural networks. IEEE Transactions on Neural Networks and Learning Systems, 33(7):2744-2757, 2020.
[65] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009.
[66] Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space, Advances in Neural Information Processing Systems, 30, 2017.
[67] Ji Lin, Chuang Gan, and Song Han. Tsm: Temporal shift module for efficient video un- derstanding. In Proceedings of the IEEE International Conference on Computer Vision, 2019.
[68] Lianghui Zhu, Bencheng Liao, Qian Zhang, Xinlong Wang. Wenyu Liu, and Xinggang Wang. Vision mamba: Efficient visual representation learning with bidirectional state space model. arXiv preprint arXiv:2401.09417 2024.
[69] Du Tran, Lubomir Bourdev, Rob Fergus, Lorenzo Torresani, and Manohar Paluri. Learn- ing spatiotemporal features with 3d convolutional networks. In Proceedings of the IEEE international conference on computer vision, pages 4489 4497, 2015.
[70] Du Tran, Heng Wang, Lorenzo Torresani, Jamie Ray, Yann LeCun, and Manohar Paluri. A closer look at spatiotemporal convolutions for action recognition. In Proceedings of the IEEE conference Computer Vision and Pattern Recognition, pages 6450-6459 2018.
[71] Zhengwei Wang, Qi She, and Aljosa Smolic. Action-net: Multipath excitation for action recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 132233, 2021.
[72] Zhaoyang Liu, Limin Wang, Wayne Wu, Chen Qian, and Tong Lu. Tam: Temporal adaptive module for video recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 13708-13718, 2021.
[73] Ze Liu, Jia Ning, Yue Cao, Yixuan Wei, Zheng Zhang, Stephen Lin, and Han Hu. Video swin transformer. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 3202-3211, 2022.
[74] Christoph Feichtenhofer, Haoqi Fan, Jitendra Malik, and Kaiming He. Slowfast networks for video recognition. In Proceedings of the IEFF/CVF international conference on computer vision, pages 6202-6211, 2019.
[75] Jiazhou Zhou, Xu Zheng, Yuanhuiyi Lyu, and Lin Wang. Exact: Language-guided conceptual reasoning and uncertainty estimation for event-based action recognition and more. In Pro- ceedings of the IEFF/CVE Conference on Computer Vision and Pattern Recognition, pages 18633-18643, 2024.
15

---

[76] Kaiming He, Xiangyu Zhang. Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings ofe IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[77] Xingjian Shi, Zhourong Chen, Hao Wang. Din Yeung, Wai-Kin Wong, and Wang-chun Woo. Convolutional Istm network: A machine learning approach for precipitation nowcasting. Advances in neural information processing systems, 28, 2015.
[78] Swathikiran Sudhakaran, Sergio Escalera, and Oswald Lanz. Gate-shift-fuse for video action recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(9):10913- 10928, 2023.
[79] Gedas Bertasius, Heng Wang, and Lorenzo Torresani. Is space-time attention all you need for video understanding? In ICML, volume 2, page 4, 2021.
[80] Zhen Xing, Qi Dai. Han Hu. Jingjing Chen. Zuxuan Wu, and Yu-Gang Jiang, Svformer: Semi-supervised video transformer for action recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 18816 18-268 2023.
[81] Lan Chen, Dong Li. Xiao Wang, Pengpeng Shao, Wei Zhang, Yaowei Wang, Yonghong Tian, and Jin Tang. Retain, blend. and exchange: A quality- spatial-stereo fusion approach for event stream recognition, arXiv preprint arXiv:2406. 18845, 2024.
[82] Yuchen Duan, Weiyun Wang, Zhe Chen, Xizhou Zhu, Lewei Lu, Tong Lu, Yu Qiao. Hongsheng Li, Jifeng Dai, and Wenhai Wang. Vision-rwky: Efficient and scalable visual perception with rwkv-like architectures. arXiv preprint wrXii:2403.023008, 2024.
[83] Yue Liu, Yunjie Tian, Yuzhong Zhao, Hongtian Yu, Lingxi Xie, Yaowei Wang. Qixiang Ye, and Yunfan Liu. Vmamba: Visual state space model. ArXiv, abs/2401.10166. 2024.
[84] Kunchang Li, Xinhao Li, Yi Wang, Yinan He, Yali Wang, Limin Wang, and Yu Qiao. Video- mamba: State space model for efficient video understanding. arXiv preprint 47X1v:2403.06977 2024.
[85] Benjamin Cramer, Sebastian Billaudelle, Simcon Kanya, Aron Leibfried, Andreas Grübl, Vitali Karasenko, Christian Pchle, Korbinian Schreiber, Yannik Stradmann, Johannes Weis. et al. Surrogate gradients for analog neuromorphic computing. Proceedings of the National Academy of Sciences, 119(4)=21091191119 2022.
[86] Nicolas Perez-Nieves, Vincent CH Leung, Pier Luigi Dragotti, and Dan FM Goodman. Neural heterogeneity promotes robust learning. Nature communications, 12(1):5791,200
[87] Julian Rossbroich, Julia Gygax, and Friedemann Zenke. Fluctuation-driven initialization for spiking neural network training. Neuromorphic Computing and Engineering, 2(4):044016, 2022.
[88] Bojian Yin, Federico Corradi, and Sander M Bohte. Accurate online training of dynam- ical spiking neural networks through forward propagation through time. arXiv preprint arXiv:2112.11234 2021.
[89] Biswadeep Chakraborty and Saibal Mukhopadhyay. Heterogeneous neuronal and synaptic dynamics for spike-efficient unsupervised learning: Theory and design principles. In The Eleventh International Conference on Learning Representations, 2023.
[90] Ziming Wang, Runhao Jiang, Shuang Lian, Rui Yan, and Huajin Tang. Adaptive smoothing gradient learning for spiking neural networks. In International Conference on Machine Learning, pages 35798-35816. PMLR, 2023.
[91] Ilyass Hammouamri, Ismail Khalfaoui- Haasaai spiking neural networks using dilated convolutions with learnable spacings. In The Twelfth International Conference on Learning Representations, 2024.
Hassani, and Timothée Masquelier: Learning delays in
[92] Sicheng Shen, Dongcheng Zhao, Guobin Shen, and Zeng. Tim: An efficient temporal interaction module for spiking transformer. arXiv preprint 111,, 2024.
16

---

[93] Shimin Zhang, Qu Yang, Chenxiang Ma, Jibin Wu, Haizhou Li, and Kay Chen Tan. Te-lif: A wwo-compartment spiking neuron model for long-term sequential modelling. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 16838-16847, 2024.
[94] Amirreza Yousefzadeh, Mina A Khoei, Sahar Hosseini, Priscila Holanda, Sam Leroux, Orlando Moreira, Jonathan Tapson, Bart Dhoedt, Pieter Simoens, Teresa Serrano-Giotarredona, al. Asynchronous spiking neurons, the natural key to exploit temporal sparsity. IEEE Journal on Emerging and Selected Topics in Circuits and Systems, ((4):668677) 2019.
[95] Mingqing Xiao, Qingyan Meng, Zongpeng Zhang, Di He, and Zhouchen Lin. Online training through time for spiking neural networks. arXiv preprint arXiv:2210.04195, 35:20717-20734 2022.
[96] Anand Subramoney. Efficient real time recurrent learning through combined activity and parameter sparsity, arXiv preprint arXiv:2303.05641, 2023.
[97] Xueyuan She, Saurabh Dash, and Saibal Mukhopadhyay. Sequence approximation using feedforward spiking neural network for spatiotemporal learning: Theory and optimization methods. In International Conference on Learning Representations, 2021.
[98] Chang Liu, Xiaojuan Qi, Edmund Y Lam, and Ngai Wong. Fast classification and action recognition with event-based imaging. IEEE access, 10:55638-5564,, 2022.
[99] Mark Schöne, Neeraj Mohan Sushma, Jingyue Zhuge, Christian Mayr, Anand Subramoney, and David Kappel. Scalable event-by event- processing of neuromorphic sensory signals with deep state-space models. Neuromorphic Computing Conference, 2024.
[100] Simone Undri Innocenti, Federico Becattini, Federico Pernici, and Alberto Del Bimbo. Tem- poral binary representation for event-based action recognition. In 2020 25th International Conference on Pattern Recognition (ICPR), pages 10426-10432 IEFE, 2021.
[101] Shu Miao, Guang Chen, Xiangyu Ning, Yang Zi, Kejia Ren, Zhenshan Bing, and Alois Knoll. Neuromorphic vision datasets for pedestrian detection, action recognition, and fall detection. Frontiers neurorobotics, 13:38. 2019.
[102] Qinyi Wang, Yexin Zhang, Junsong Yuan, and Yilong Lu. Space-time event clouds for gesture recognition: From rgb cameras to event cameras. In 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 1826-1835. IEEE, 2019.
[103] Wei Fang, Zhaofei Yu, Yanqi Chen, Timothée Masquelier, Tiejun Huang, and Yonghong Tian. Incorporating learnable membrane time constant to enhance learning of spiking neural networks. In Proceedings of the IEEE/CVF International Conference Computer Vision (ICCV), pages 2661-2617 2021.
[104] Yansong Peng, Yueyi Zhang, Zhiwei Xiong, Xiaoyan Sun, and Feng Wu. Get: Group event transformer for event-based vision. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6038-6048, 2023.
[105] Ze Liu, Han Hu, Yutong Lin, Zhuliang Yao, Zhenda Xie, Yixuan Wei, Jia Ning, Yue Cao, Zheng Zhang, Li Dong, et al. Swin transformer v2: Scaling up capacity and resolution. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12009, 2022.
[106] Lin Zuo, Yongqi Ding, Wenwei Luo, Mengmeng Jing, Xianlong Tian, and Kunshan Yang. Temporal reversed training for spiking neural networks with generalized spatio-temporal representation. arXiv preprint 44Xiv:2408.09 2024.
17

---

A Supplementary Section A: Detailed Proofs
A.1 Computational Complexity of Event-Driven SSMs
Lemma 1, Let the event-driven state-space model be governed by:
=(t) Ax(t) BS(().
where æ(t) E R is the internal state, E R N*N is the state transition matrix, and S(t) E RM is the imput event train. The compulational complexity of updating the internal state x(()) each event is O(N²).
Proof. The event-driven state-space model is governed by:
i(t) - Ax(()= + BS((),
where sx(1 IR represents the internal state of the system, A RRNx is the state transition matrix, and S(t) R represents the input event train. When event occurs at time t,, the state update can be represented by the following integral equation for E [4,, tt:)::
x = = ( S.a Aet) ( dr,
where:
1 and i are the times just before and after the event at 4,
At t is infinitesimal,
S(T) contains Dirac delta functions at event times and is zero elsewhere.
For simplicity, we focus on the update at the event time 4 to approximate the state transition each event.
The update of the internal state x(t) requires computing the matrix exponential Adt where At = t- represents the time interval between successive events. Computing the exact matrix exponential for general matrix A N is computationally expensive, involving O(N3) operations using standard algorithms such as diagonalization or the Schur decomposition.
To reduce the computational cost, we approximate the matrix exponential using truncated Taylor series expansion:
ADL, I A&I, A2
where / the identity matrix of size N N. This approximation typically sufficient for small At, which is common between events.
In the Taylor series expansion approximation of AAt the dominant computational cost arises from multiplying the matrix E R. NxN by itself and by the state vector <<t) N...
The product Ar(t). where computational cost for this step is O(V²).
NxN and xxt) N requires 2² multiplications. Thus, the
The term _A is computed by multiplying A by itself. Since A is N x N matrix, computing A² explicitly would have a computational cost of O(N*). However, we avoid this by computing A( A(Ar(t)), which involves two sequential matrix-vectorr products, each costing O(N2). Therefore, the computational cost of computing A A²x(1) )) is O(N2).
The term BS()), where B NxM and S(t) RM, involves Q(NM) operations. Assuming Al is proportional to N or smaller, this computation contributes O(N²) ) to the overall complexity.
To update the internal state z(t), perform the following operations: First, multiply A by (tt): O(N2); then multiply A² by x(t): O(N?): followed by multiplying B by S(t): O(NM) and finally add the resulting vectors.
Thus, the overall computational complexity for updating the internal state x(!) at each event is O(N2).
18

---

In the general case, where A is dense matrix, the cost of updating the state is O(N2). If the matrix has a specific structure. such as being sparse or diagoniag,n,,, the computational cost can be reduced. For example: If A is sparse with k non-zero entries per row, the cost of multiplying A by s(t) becomes O(kN), which can be significantly lower than O(N2) when < N. If A is block-diagonal, the cost can be reduced to O(N) per block, depending on the number and size of the blocks. However, for the general case where no such structure assumed, the computational complexity remains 2 The computational complexity of updating the internal state xx() at each event, using the matrix exponential approximation with a Taylor series expansion, is dominated by the matrix-vecter multiplication operations. Additionally, accounting for the BS(t) term maintains the overall complexity at O Therefore, the overall computational complexity for updating the internal state at each event is ((N3).
A.2 Long-context Temporal Dependency Preservation Via Event-Based Hippo Theorem 1. Let x(() e RN evolve according to
=(t) = Ax(t)+
where: A NNN is HiPPO matrix with all eigenvalues satisfying Re(A,) 0 i= i,, ..,.....N. N, B RNM is the input matrix, S(t) RM is the input event train, assumed to be bounded, i.e., there exists constant Sx O such that [S((]] s S for all 0, Ix x<0) TR is the initial state.
Then, the event-driven SSM preserves long-context temporal dependencies in the input event train S(t), and the state xt) satisfies the bound:
|B| (1- (1-e-at).
[x(()] se se""( [
where min; [Re(A:)) 0 is the memory retention factor determined by the eigenvalues of the HiPPO matrix A.
Proof. To establish the theorem, will analyze the evolution of the internal state sr() governed by the differential equation:
a(t) Ax(t) BS(().
with initial condition z(0) = Zg-
The differential equation is non-homogeneous linear ordinary differential equation (ODE). Using the variation of parameters method, the solution can be expressed as:
xx EA S. ((LT)) BS( dr,
where: - 4t Fn the solution to the homogeneous equation att) = Ax(t) with initial condition xz(0)= =o. fo BS(+) dr accounts for the particular solution due to the input S(t).
Given that A is HiPPO matrix, all its eigenvalues satisfy Re(A:) 0 for i 1,2,..., N. This implies that A is a Hurwitz matrix, ensuring that the system is asymptotically stable. Define the memory retention factor
= min Re (e(A))) 0.
This factor dictates the rate at which the influence of the initial state xp decays over time.
Consider the homogeneous solution A Since all eigenvalues of A have negative real parts, the matrix exponential At satisfies:
| se
where |||| denotes an operator norm (e.g., the induced 2-norm). This inequality leverages the spectral bound of A to provide an exponential decay rate.
Therefore, the contribution of the initial state is bounded by:
Ae æol s le
|| s se s=**((zo))
19

---

Next, consider the particular solution:
S.¹ ((lt dr.
To bound its norm, apply the triangle inequality and properties of operator norms:
Iso ell-T) BS(() dr| L. || |B [[(T)) dr.
Given that |S(r)) s So and le (e s a((t)) we have:
1.,' A((-T) (S(T)) BS( dr s B| Jo c-((1))) dr.
Evaluate the integral:
1 e-t Jo" c-alttr) dr= S/' c-as ds
a
Thus, the bound becomes:
I1o e ((())) BS(T) dr
|B| (1- e cate
a
Combining the bounds for the homogeneous and particular solutions, obtain:
||x(t||| |e<ooll |.6'' (((+=)) BS(T) dr| se se"oo
| (1-e-at).
This inequality demonstrates that the influence of the initial state zo decays exponentially rate a. Also, the accumulated influence of the input event train S(t) is bounded and grows to steady-state value determined by B||, Soe. and
The derived bound:
[xx(t] S se s [BIS
(1 eat)
reveals that the term e e- ||aul signifies that the system "forgets" its initial state exponentially fast, ensuring that old information does not dominate the state indefinitely. Also, the integral term captures the accumulated influence of the input event train S(1). Since S(I) bounded, the state xx() can retain and reflect information from the input over extended periods without being overwhelmed by the initial condition.
Therefore, the event-driven SSM governed by a HiPPO matrix A effectively preserves long-context temporal dependencies in the input event train S(t), while ensuring that the memory of the initial state To decays at an exponential rate determined by a.
A.3 Error Bound For Event-Driven Matrix Exponential Approximation
Lemma 2. Let the matrix exponential be approximated using a Taylor expansion up to the n-th term:
A!2
AL
AAt AAt
2!
n!
Assume that the matrix norm
|-|| is submultipicaaive, iee. |AB| s [A| B for any matrices A and B of compatible dimensions. Then, the error E,, of this approximation satisfies
|A MAN
|E
(n+
Proof. The matrix exponential can be expressed as an infinite Taylor series:
(ADI)*
AAr
Σ
h!
k=0
20

---

If we truncate this series after the 12th term, the remainder E, is given by:
(AA
(AAr)*
En AA
Σ
M!
k=n+1
To bound the norm of the error Ens we apply the submultiplicative property of the matrix norm:
20
(AAt)*
20
|
ExI
Σ
Σ
kd
k!
k=++1
Using the submultiplicative property of the matrix
20
LAV/ R
|| En
S
Σ
h!
k=++1
Let = ||At|] O. Then:
E Σ k!
Since
Σ
Rn(
K!
k=n+1
where (2) is the remainder of the Taylor series expansion of e*
According to Taylor's Remainder Theorem (Lagrange's 's form), there exists & E [0, æ] such that:
n++
(x)
(n 1)! es.
Since & x and et s e for x 2 O, we have:
x++
Rx(x))
(n + 1)!
Therefore:
n+1
|AAt||=+ CAA
Ex 5
(n+ 1)!
(n + 1)!
Thus, the error En satisfies:
|AA/|1-11 clAnd
| [.|| S
(n+ 1)!
A.4 Boundedness Of State Trajectories With Event Inputs
Theorem 2. Boundedness State Trajectory in Event-Driven State-Space Models
For given initial condition Fo, the state trajectory x(1) of the FLAME model driven by the event input S(t) is bounded, i.e., ||x(t]] C, for some constant C > 0, provided that:
21

---

1. The input events (t) are of finite magnitude, i.e. |S(t)]l for all t 0.
2. The decay matrix As is Hurwitz, meaning all its eigenvalues have negative real parts.
3. There exists a positive definite matrix P satisfying the Lyapunov equation A P As = Q, for some positive definite matrix Q
Proof. Consider the FLAME governed by:
a(t) = Asx(t) + BS(t),
where As a Hurwitz matrix, B is the input matrix, and St)) is bounded input event train with [S(t)] & S. for all 2 O.
We define a Lyapunov function V(x) = x² Px, where P is a positive definite matrix satisfying the Lyapunov equation:
P PAs=
with Q being a positive definite matrix. Such a / exists because As is Hurwitz. The derivative of (ar) along the system trajectories is computed:
d
V(z)= (x Pz) Pz)=="² = x x Ph Px.
Pz)
Since P is constant (P- = 0), and ã Agx BS((), this simplifies to:
(x) x" P(Agæ + BS(()) (Agæ + BS())))Pz.
Recognizing that symmetric (P" P), we can write:
(xx) = x [AT PAs)x 2x PBS(t).
Substituting the Lyapunov equation P PAs =Q:
v(x)= - Qx 2x PBS(t).
The term 2rS(t bounded using the Cauchy- SchuhrySShwarz inequality
2x PBS(t) ||| |PB| |S(t] P|
Next, let us define = P The derivative (x) becomes:
5-=) +
Since Q is positive definite, ==Ou 2 Ami () 1x where Amin (Q) is the smallest eigenvalue Q. Therefore:
v(x) s -Aeiin - (2)
7//2/|
Completing the square:
2 )²
2²
((x) -Ammm (Q) |-||2- Amiin (Q)
( [0l --mmin 2mmin (Q))
4Amin (Q)
This inequality indicates that V(x) whenever |a| (Q) Since V(x) and V(z) is
negative outside ball of radius C (2))(Q)) the state x(t) will ultimately remain within this bounded region. Therefore, [x(t)] s C for all f2 0
22

---

B Supplementary Section B: Extended Experimental Results
B.1 Datasets and Tasks
In this study, we evaluate the performance of the FLAME model across a diverse set of datasets, each presenting unique challenges in event-driven processing. The datasets include Sequential CIFAR-IO, Sequential CIFAR-100 [55] DVS Gesture (7]. HAR-DVS [6]] Celex-HAR (2). Spiking Heidelberg Digits (SHD) [64], and Spiking Speech Commands (SSC). For all experiments. the FLAME model processes inputs on an event-by-event basis, leveraging its temporal dynamics to handle fine-graine temporal dependencies without accumulating events into frames. Below, provide detailed descriptions of each dataset and the corresponding experimental setups.
Sequential CIFAR-10 and CIFAR-100. The CIFAR-10 10 and CIFAR- 100 datasets 65 consist of 32 32 RGB images across 10 and 100 classes, respectively. To simulate temporal sequence, each image is divided into 16 non-overlapping patches of size 8 x 8 pixels. These patches are presented to the model sequentially in a raster-scan order, from top-left to bottom-right. Each patch is treated as independent event in the sequence. The task involves classifying the image based on the full sequence of patches, requiring the model to integrate information over the entire sequence. This setup evaluates the model's ability to process spatial information in temporal context.
This dataset comprises recordings of 11 hand gestures performed by 29 subjects under varying lighting conditions, captured by Dynamic Vision Sensor (DVS). Event streams are processed at a resolution of 128 x 128 pixels. Sequences typically span approximately 1-6 seconds, and the maximum number of events processed per sequence is approximately 98k. FLAME processes each event individually as occurs, without frame accumulation, to preserve high temporal resolution. The events in this dataset are highly sparse and asynchronous, with non-uniform distributions over the sequence duration.
DVS Gesture Dataset 77:
HAR-DVS Dataset [61]: The HAR-DVS dataset contains neuromorphic event streams representing human activities, such as walking and running. These sparse, asynchronous streams are recorded at resolution of 346x x 260 pixels. Sequences generally last around 5 seconds, with the maximum number of events processed per sequence reaching approximately 450k. Our model processes each event-by-event dynamically updating its internal state for each incoming event to precisely model activity sequences. The events are characterized by their spatial coordinates, timestamp, and polarity, exhibiting high sparsity and asynchronous, non-uniform temporal distribution.
This dataset consists of high-resolution (1280 800 pixels) event streams of human actions like sitting, standing, and walking, captured with a CeleX camera. Sequences mostly last around 2-3 seconds, with some extending beyond seconds. The maximum number of events processed per sequence is up to approximately 3.1 million. FLAME processes each event- by-event, enabling it to capture the fine- nneeggained temporal dynamics of these high-resolution human activities. Similar to other event-based datasets, the event data is highly sparse, asynchronous, and non-uniformly distributed over the sequence duration.
Celex-HAR Dataset [2):
Spiking Heidelberg Digits (SHD) and Spiking Speech Commands (SSC): The SHD and SSC datasets (64) contain neuromorphic event streams derived from speech datasets. SHD consists of spoken digit recordings converted to event trains using the CochleaAMS model, while SSC contains event-based representations of spoken command audio, representing keywords like "yes," " "no." " and "stop." Each event is characterized by its spatial location, timestamp, and polarity. The datasets evaluate the model's performance on tasks involving complex spatio-temporal patterns speech data, The FLAME model processes each event as it occurs, dynamically updating its state, ensuring high temporal resolution and efficient processing for speech recognition tasks.
Across all datasets, the FLAME model processes inputs on an event- event basis. This approach allows it to maintain high temporal resolution and capture fine-graine spatio-temporal patterns, distinguishing it from frame-based methods. The event-by-event design also reduces computational overhead and ensures low latency, making the model well-suited for real-time applications.
B.2 Ablation Studies:
To evaluate the contribution of individual components the FLAME model, we performed extensive ablation studies on the sequential 11 dataset. Specifically, we analyzed the impact of removing
23

---

Table Comparison of FLAME models with state-of-the-at on the HARDVS dataset. Accuracy is measured in percentage, and computational cost is in GFLOPs.
Model
GFLOPs
Accuracy (%)
C3D 669
0.1
50.52
R2PlusID
70
20.3
49.06
TSM 67
0.3
52.63
ACTION-Net
17.3
46.85
TAM 772
16.6
50.41
V-SwinTrans
8.7
51.91
SlowFast 74
0.3
46.54
ESTF [6]
17.6
51.22
ExACT 775
1.3
90.10
FLAME-TTnY [Ours]
0.034
65.42
FLAME- MAAME-Small [Ours]
0.13
79.36
FLAME-Normal [Ours]
0.41
88.29
Table 4: Detailed Architecture of FLAME Models (Tiny, Small, and Normal)
Layer Type
FLAME Tiny
FLAME Small
FLAME Normal
Input Representation
Asyachronous Event Events (p.W.L.p)
attention brunches
32 attention branches
64 attention branches
Event Atfention Layer
72 = [T Tie]
Ty [T].... T32]
Tg [T Tee]
Conv2D (32 filters, 3x3)
Conv2D (64 fitter, 3x3)
Conv2D (128 filters, 3x3)
Convolutional Block 1
Batch Norm. Max Pool (2x2)
Batch Norm, Mux Pool (2x2)
Batch Norm. Max Pool (2x2)
Conv2D (32 filters, 3x.3)
Con-2D) (64 filters, 3x.3)
Conv2D (128 fitter, 3x3)
Convolutional Block 2
Batch Norm, Pool (2x2)
Norm, Pool (2x2)
Batch Norm, Pool (2x2)
Spatial Pooling Layer
Pool (2x2)
Pool (2X2)
Pool (2x2)
FLAME Convolution
State Update using Event-Aware HiPPO and NPLR decomposition for efficient event- convolation Layer Norm
Layer Norm
Layer Norm
Normalization Layer
Normalizes the state variables to stabilize training
Fully Connected (256 neurons)
Fully Connected (512 neurons)
Fully Connected (1024 neurons)
Readout Layer
Softmax for classification
Softmax for classification
Softmax for classification
or replacing key components such the event attention layer, Ewent-Aware HiPPO (EA-HiPPO) NPLR decomposition, and FFT convolution. The results of these experiments, along with the corresponding model parameters and computational costs (in GFL.Oss), are summarized Table 7
Impact of Event Attention Layer (EAL) Removing the event based attention mechanism leads to a reduction in both accuracy and model parameters. The accuracy drops across all channel configurations, with the largest channels (128) seeing a decrease from 90.25% to 85.83%. The smaller channel configurations (64 and 32) experience similar drops, highlighting the event based
Convolutional Models
SSN Besed DWK Mode's
SNN Models
Proposed Models
SHD
SSC
DVS
Eeea 2010
FLAMES [Duns]
FLAMES (128) [Ours]
BrA (Dhngg 2000
Accuracy (96)
2H-RRN 2hang2oon
DOLD
(84).
A
FLANES
L3TH
[Ours)
pant
Ponskvos13322 1222
2930
WWw
(ha))
Vit Parameters (M)
Parameters (M)
Parameters (M)
Figure 4: Comparison of our FLAME to the state-of-the-art on BVS128-Gesture Spiking Heidel- berg digits (SHD) and Spiking Speech Commands (SSC) [64 datasets
24

---

Table 5: Experimental results CeleX-HAR dataset.
No.
Algorithm
Publish
Arch.
FLOPs
Params
acc/top-1
Code
01
ResNet-50
76
CVPR-2016
CNN
8.6G
11.7M
0.642
URL
02
ConvLSTM
NIPS-2015
CNN, LSTM
0.539
URL
03
C3D 69
ICCV-2015
CNN
0.IG
147.2M
0.630
URL
R2
CVPR-2018
CNN
20.3G
63.5M
0.679
URL
TSM
ICCV-2019
CNN
0.3G
24.3M
0.704
URL
06
ACTION-Net
71
CVPR-2021
CNN
17.3G
27.9M
0.685
URL
07
TAM 72
ICCV-2021
CNN
16.6G
25.6M
0.705
URL
08
GSF 78
TPAMI-2023
CNN
16.5G
10.5M
0.703
URL
V-Swin Frans
73
CVPR-2022
VIT
8.7G
27.8M
0.689
URI
10
TimeSformer
79
ICML-22011
VIT
53.6G
121.2M
0.680
URT
11
SlowFast 74
ICCV-2019
VIT
0.3G
33.6M
0.680
URL
12
SVFormer 80
CVPR-2023
VIT
196.0G
121.3M
0,610
RRL
EFV++
arXiv-2024
VIT. GNN
36.3G
39.2M
0.695
14
ESTF [6]]
AAAI-2024
VIT. CNN
17.6G
46.7M
0.673
15
VRWKV-S 82
arXiv-2024
RWKV
4.6G
23.8M
0.661
JRL
16
VRWKV-B
82
arXiv-2024
RWKV
18.2G
93.7M
0.668
URL
Vision Mamba-S
68
ICML-20244
SSM
5.1G
26.0M
0.701
RRL
18
VMamba-S
83
arXiv-2024
SSM
11.2G
14.7M
0.713
URL
19
VMamba-S(VV) ((2)
83
arXiv-2024
SSM
8.7G
50.4M
0.715
URL
20
VMamba-B
83
arXiv-2024
SSM
18.0G
76.5M
0.720
URL
VMamba B(V2)
86
arXiv-2024
SSM
15.4G
88. 98.9M
0.718
22
VideoMamba-S
84
ECCV-2024
SSM
4.3G
26.0M
0.669
URL
23
VideoMamba-X -
84
ECCV-2024
SSM
12.7G
74.0M
0.691
RL
24
EVMamba
arXiv-2024
SSM
37.2G
76.5M
0.723
URL
25
EVMamba w/o Voxel Scan
arXiv-2024
SSM
18.0G
76.5M
0.720
URL.
26
FLAME-Tiny (Ours)
SSM
0,034G
7.91M
0.632
27
FLAME-Smal (Ours)
SSM
0.13G
13.35M
0.692
28
FLAME-Normal (Ours)
SSM
0.41G
25.57M
0.722
attention's 's role improving the spatio-temporal feature representation. Interestingly, removing this mechanism slightly reduces the model's GFLOPs since the computations associated with the event attention layer are avoided.
Impact of Event-Aware HiPPO Replacing EA-HiPPO with a simple LIF-based mechanism leads to moderate drop in accuracy (e-g., from 90.25% 87.62% for 128 channels). However, this modification does not alter the computational cost (GFLOPs), as EA-HiPPO primarily affects the temporal memory adaptation rather than the core matrix or convolution operations. These results emphasize EA-HiPPO s critical role in retaining and managing temporal dynamics effectively.
Impact of NPLR Decomposition The NPLR decomposition significantly reduces the computational complexity of state-space updates. For the FLAME-Normal (128 channels) model, removing NPLR also resulted in notable increase in GPU memory usage by approximately 38% (from 5,00 MB 6.91 MB), confirming NPLR's ' essential role in minimizing both computational cost and memory footprint for real-time deployment. Despite this computational overhead, the accuracy remains relatively stable, highlighting that NPLR's primary advantage is computational efficiency rather than feature extraction performance.
Impact of FFT Convolution FFT convolution integral to efficiently handling long-context temporal dependencies. Replacing FFT convolution with standard time-domain convolution increases the GFLOPs substantially (e.g., from 0.43 GFLOPs to 1.2 GFL.PPs for 128 channels). Furthermore, the accuracy
more pronounced decline (e.g., from 90.25% 86.47%), particularly in tasks requiring high temporal resolution. These results underscore FFT convolution's 's dual role in reducing computational cost and maintaining temporal modeling performance.
Summary of Findings The ablation studies validate the critical importance of each component the FLAME model:
The event attention layer enhances the spatio-temporal feature representation, significantly improving accuracy.
25

---

Table Comparison of classification accuracy and parameters of different models across SHD and SSC datasets.
Model
SHD
SSC
#Parameters
Accuracy (%)
#Prrameters
Accuracy (%)
SFNN
0.09 M
48.1
0.09 M
32.5
SRNN
64
1.79 M
83.2
SRNN
85
0.17 M
81.6
SRNN
0.11 M
82.7
0.11
60.1
SCNN 87
0.21 M
84.8
SRNN 88
0.14 M
90.4
0.77 M
74.2
HRSNN
89
80.01
59.28
LSTM 64
0.43 M
89.2
0.43 M
73.1
DH-SRNN 33
0.05 M
91.34
0.27
81.03
DH-SENN 33
0.05 M
92.1
0.35 M
82.46
ASGL 90
78.90
78.90
DCLS 91
0.2
95.07
2.5
80.69
TIM 92
2.59 M
86.3
0.111 M
61.09
TC-LIF 93
0,142 M
88.91
FLAME-Normal (128) [Ours]
0.513 M
94.68
0,513 M
87.52
FLAME-Small (64) [Ours]
0.129 M
90.57
0.129 M
82.08
FLAME-Tiny (32) [Ours]
0.033 M
86.24
0.033 M
72.19
EA-HiPPO dynamically adjusts temporal memory retention, contributing to performance robustness without additional computational overhead.
NPLR decomposition ensures scalability by reducing the computational cost of state-space updates, making the model efficient for large-scale tasks.
FFT convolution is indispensable for capturing long-contex, dependencies efficiently while keeping computational complexity low.
B.3 DVS Gesture Recognition
To further investigate the combined effectiveness of Event-based Attention mechanisms and FLAME convolutions event- based processing. we evaluate our model on the DVS Gesture dataset. This dataset consists of event streams recorded from Dynamic Vision Sensor (DVS) at resolution of 128x 128, providing a challenging benchmark evaluating temporal dynamics gesture recognition tasks involving varying speeds and motions.
Our goal is assess how the integration of event-based attention mechanisms with FLAME convolu- tion layers enhances the model's ability to capture multi-scale temporal dependencies. Specifically, we examine how event-based attention can serve a temporal attention mechanism that helps FLAME effectively focus on the most relevant events, while FLAME convolutions manage the overall temporal and spatial evolution of features.
The experiment involves training variants our model-one one incorporating both event- atten- tion mechanisms and FLAME convolutions, and the other using only FLAME-E determine the contribution of event-based attention [ablee7/summarize the test accuracy of our models compared other state-of-the-art approaches. The results measured in terms of classification accuracy, along with the number of parameters, to highlight model efficiency.
As shown in Table 7 the FLAME model with 128 channels, incorporating event-based attention. achieves 96.5% accuracy while maintaining a significantly lower parameter count compared to many other state-of-the-an models. This shows that approach effectively utilizes sparse event-driven inputs to achieve high accuracy with reduced computational complexity. The use event- based attention mechanisms allows the model to dynamically adjust its focus different temporal scales, thus improving gesture recognition even in scenarios with rapid motion changes.
The variant without event-based attention. while still competitive, lags behind in adapting to the multi-scale nature of the event data, especially for gestures with complex temporal characteristics. This indicates that the event-based attention mechanism plays a crucial role in adaptively filtering
26

---

Table Comparison of classification accuracy, parameters, and FLOPs of different models across
the DVS128 Gestuee dataset.
Model
Parrameters (M)
GFLOPs
Accuracy (%)
Yousefzadeh al.
94
1.2
95.2
Xiao el
96.9
RTRL 996
4.8
97.8
She et al.
1.1
98.0
Liu et al. 98
98.8
Chakraborty
2F
96.5
Martin-Turrers
15
14
96.2
Martin-Turrero ot
14
94.1
CNN + (time-frames)
99
6.8
97.8
Event-SSM
97.7
CNN + S5 (event-frumes
59
6.8
97.3
TBR+13D 1000
12.25
38.82
99.6
12.37
96.5
EV-VGCNN 13
0.82
0.46
95.7
RG-CNN [0]
19.46
0.79
96.1
PointNete-4 102
1.48
0,872
95.3
PIJF [03
1.7
97.6
GET [004
4.5
97.9
Swin-T
105
7.1
93.2
TIPOINT
0.334
0.587
98.8
EventMamba 40
0.29
0,219
99.2
STC-LIF [06]
3.922
83.0
Spiking Transformer
36.01
33.32
FLAME-Normal (128) [Ours]
0.513
0.43
96.5
FLAME (64) [Ours]
0.129
0.14
93.7
FLAME-Timg -anyTTyy (32) [Ours]
0,033
0.07
89.2
FLAME-Norma (128 Channels) No Attention Branch [Ours Ablation]
0,501
0.43
95.2
TLAMFF-Smia (64 Channels) No Attention Branch [Ours Ablation]
0.121
0.14
89.3
FLAME-Tiug (32 Channels) No Attention Branch [Ours Ablation]
0.031
0.07
81.5
FLAME- -ormal (128 Channelk) No HiPPO [Ours - Ablation)
0,501
0.43
90.4
FLAME-Sman Small (64 Channcls) No HiPPO [Ours Ablation]
0.121
0.14
82.6
FLAME: -uny (32 Channels) No HiPPO [Ours Ablation]
0.031
0.07
73.5
relevant temporal features, which is essential for handling the asynchronous, irregular inputs typical of event cameras.
In addition, visualizations of the learned event-based attention reveal how the model attends to different time segments. effectively filtering the incoming event streams to prioritize the most relevant events. This adaptive filtering complements the FLAME convolutional operations, leading to more robust and efficient temporal feature extraction.
Overall, the results validate the utility of combining event-based attention mechanisms with FLAME convolutions for event-driven tasks, making the model well-suited for gesture recognition from DVS inputs. The joint use of these components allows for efficient temporal modeling, maintaining favorable trade-off between accuracy and parameter efficiency.
B.4 Scaling to HD Event Streams
The scalability the proposed FLAME model is evaluated on the Celex HAR dataset, a human activity recognition dataset recorded at high resolution of 1280 800. This dataset serves challenging benchmark for assessing the model's ability to maintain high accuracy and computational efficiency when processing large-scale spatial and temporal data.
In this experiment, FLAME used for action recognition on HD event streams, and its performance compared to that of baseline Spiking Neural Networks (SNNs) and State State-Space Models (SSMs). As shown in Figure the results demonstrate that FLAME maintains high accuracy even at increased resolutions, whereas the baseline models experience significant performance degradation due to heightened computational demands. The integration of the FLAME convolution layer proves effective in managing the complex spatial and temporal components of HD event data, providing robust real-time processing capabilities with minimal computational overhead.
Figure illustrates the trade-off between accuracy and computational cost, measured in terms of FL.PSs for our FLAME models compared to state-of-the-art methods on the Celex-HAR dataset. The
FLAME variants-FLAME Tiny, FLAME Small, and FLAME Normal-demonee demonstrameeee superior effi-
27

---

Table 8: Latency Comparison on Celex-HAR (in milliseconds)
Algorithm
Latency (ms)
Algorithm
Latency (ms)
Algorithm
Latency (ms)
FLAME-Tiny
0.162
TAM
76.012
VideoMamba-S
19.707
FLAME-Small
0.582
GSF
75.558
VideoMamba-M
58,164
FLAME-Normal
1.867
V-SwinTrans
39.837
EVMamba
170.34
ResNet-50
41.575
TimeSformer
255.425
EVMamba w/o Voxel Scan
82,423
C3D
0.473
SlowFast
1.118
VRWKV-S
21,091
R2Plus1D
94.264
EFV++
166.23
VRWKV-B
86.346
TSM
1,4266
ESTF
80.61
Vision Mamba-S
23.88
ACTION-Net
81.035
SVFormer
897.455
VMamba-S
53,302
TAM
76.012
VMamba-S(VV
39.848
GSF
75.558
VMamba-BB B
82,421
V-SwinTrans
39.837
VMamba-B B(V2)
70.514
ciency by achieving competitive or better accuracy while utilizing significantly fewer computational resources. Key observations from Figure 2 are follows:
Efficiency at Different Scales: FLAME Tiny achieves approximately 63.8% accuracy with fraction the computational cost compared larger models such as SlowFast and C3D. As the model scales to FLAME Smull and FLAME Normal, accuracy improves to 69.3% and 72.1 %, respectively, while maintaining a favorable computational cost profile.
Performance with Reduced Complexity: FLAME Normal matches or exceeds the accuracy of models like TSM and VisionMamba-S but at substantially lower computational cost. This efficiency is attributed to the integration of event-driven processing and effective state-space dynamics.
The improved efficiency of FLAME can be credited to the event-based processing capabilities the FLAME architecture and the FLAME convolution layer, which optimally manage state-space evolution without relying on dense operations. These features allow the model to capture complex temporal dependencies while minimizing computational requirements, making FLAME particularly effective for high- higltresolttion event-based datasets like Celex-HAR.
The HAR-DVS dataset results underscore the advantages of our FLAME models, achieving accuracies of 70.38 % 81.73% %, and 88.29% for FLAME- Tiny, FLAME-SSalll and FLAME-Normal, respectively, while maintaining substantially lower computational costs compared to other of models. Unlike traditional deep neural networks such as C3D and R2Plus which struggle model the complex temporal relationships inherent in event streams, FLAME leverages a novel event-by-evenl processing approach, preserving fine-grained temporal dynamics essential for accurate action recognition.
HAR-DVS Results:
Moreover, FLAME employs a unique event-base attention mechanism that enhances its ability to capture long-context ppatio-temporal dependencies efficiently. The prolonged and complex actions in HAR-DVS demand robust temporal attention mechanisms, as highlighted in prior studies. FLAME's dendritic-inspired design meets these requirements while offering computationally efficient solution, making particularly suitable for real-time, low-latency applications in dynamic event-driven environments.
important to note that HAR-DVS provides frame-based data, as raw event data was unavailable for download. Since FLAME is designed for event-by-event processing, we treated all events arriving the same timestamp as single batch for processing, adhering to the event-driven principles of the model.
28

---

B.5 Performance on N-Caltech101
To further assess the generalization capabilities of FLAME on object recognition tasks using event- based data, we evaluated its performance on the N-Caltech101 dataset. The results for FLAME-Tiny, FLAME-Smal and AAME-Nommal are presented in Table
Table 9: Performance of FLAME Variants on N-Caltech 101.
Model Variant
Accuracy (%)
Params (M)
FLOPs (G)
FLAME-Tiny 65.9
0.033
0.71
FLAME-Smal
68.3
0.129
0.86
LLAAME-Norml 70.5
0.513
1.34
The results demonstrate that FLAME scales effectively with model capacity this challenging classification benchmark, achieving competitive accuracy while maintaining a low computational footprint. This reinforces its suitability for real-world event-based learning scenarios involving diverse object categories.
B.6 Hardware Deployment and Efficiency Profile
To assess the practical deployability and efficiency of FLAME beyond standard GPU benchmarks, profiled the FLAME-Tiny variant across a diverse range hardware platforms. This evaluation aims demonstrate FLAME' adaptability both high-performance computing environments and more resource-constrained settings, which is crucial for real-world applications of event-based systems. The FLAME-Tinn model was selected for this profiling due its design emphasis minimal computational footprint, making it suitable candidate for edge deployment scenarios.
The performance of FLAME-Tiny, in terms of inference accuracy and latency, was measured on several common hardware backends, including a standard CPU and various NVIDIA GPUs. The results, obtained without any model retraining specifically for these hardware targets, summarized in Table
Table 10: FLAME-Tiny Inference Performance Diverse Hardware Platforms. Accuracy was
evaluated on the Celex-HAR dataset. Latency is reported in milliseconds (ms).
Hardware
Aceuracy (%)
Latency (ms)
Intel Xeon CPU (2 vCPUs @2.2GHz)
63.2
42.1
NVIDIA T4 GPU
63.2
1.57
NVIDIA V100 GPU (32GB)
63.2
0.43
NVIDIA A100 GPU (40GB)
63.2
0.162
The results in Table TO indicate that FLAME-Tiny maintains accuracy (63.2% on Celex-HAR) consistently across all tested hardware. As expected, the inference latency varies significantly, with the NVIDIA A100 GPU providing the fastest processing time at 062 ms. Even on a CPU, FLAME- FLAME-Tiny achieves reasonable latency of 42.1 ms, demonstrating its potential for deployment in environments where specialized GPU hardware may not be available. The performance on the NVIDIA T4, GPU often used for inference in datacenter and edge scenarios, also shows a practical latency of 1.57 ms.
These findings support the claim that FLAME, particularly its lightweight variants like FLAME-Tiny - can efficiently deployed across a spectrum of computational platforms, from high-end GPUs to more seneral-purpose CPUs, without compromising accuracy. This versatility is a key advantage for event-based processing, where applications can range from power-constrained mobile devices to high-the tigh-throughhut server-side systems.
29

---

C Supplementary Section C: Methods and Architectural Details
Algorithm FLAME Model Training
Require:
Training dataset D (X),, learning rate 7), total epochs E, threshold potential
Va, decay factors Ola. _
I: Initialize weights W, event-based attention timing factors Ta, FLAME matrices A, B, C, low- rank matrices P. Q, and kernel ((w)
2: Initialize coupling strengths ga for each attention branch d
3: for epoch to E do
4: for each yX 2D do
Input Representation: Prepare input events for processing
5:
Parse input event sequence X { 44, pt)), where (4) spatial coords, is
time, P; is polarity.
Event Attention Layer: Update attention branch currents and aggregate at soma for each & in event sequence do
6:
7:
for each attention branch do
8:
Update attention branch current: ia(t; + aag ia(t:) + Ejar, Wi end for
9:
10:
Aggregate currents at soma: V(4 + 3 V(t)) (14))++ Exga ii(()) if +1; Vn then
11:
12:
Generate event and reset potential: V(4 +-0 end if end for
13:
Spatial Pooling Layer: Reduce spatial dimensionality while preserving temporal resolution
15:
Apply max pooling: ppalled (x', y'.yttt) (a)(a.))ePe'' (z,y,t)
FLAME Conv. Layer: Apply EA-HiPPO, NPLR, & FFT for event dynamics Initialize state vector x(0)
16:
17:
for each event time &u in Ipooted do
18:
Compute At,- t - tg. decay F (Atx) e- A.y
19:
Compute event aware HiPPO: As A F(Att))
20:
Decompose: As = VAV* - -PQ"
e Ashta & I AsA + As 2 (A4))
21:
22:
Update: x((x.1) As x(1x) As (eAsSu -I)-BS(()) S(4))
23:
FFT-based convolution: x(4++1)
(w)) FFF(((((+1))
24:
end for
25:
Compute continuous output: y()) x(1()
26:
Thresholding: Convert (t) to events by applying Devent (1) T(y(t) Vín)
Normalization: Reduce variability in activations
27:
Apply layer normalization: &
Vatte ?
Readout Layer: Compute final output and update model parameters
28:
Compute pooled state: Xpooled,
Siokp
(k-1)p-1 x;
29:
Final output: Ypred W Xpooled b
30:
Compute loss pyyrrd, update W W Wn ac
3w
31:
end for
32:
end for
Background and Preliminaries
State-Space Models: A state-space model (SSM) is a mathematical framework for modeling systems that evolve over time. The dynamics of such systems are described by set of first-order differential equations, often expressed in continuous time as:
z(t) Ar(t)++ Bu(t), CC()) Du(t)
=
where:
=(() N is the hidden state vector, representing the internal state of the system at time t,
30

---

utt) RM is the input signal, such as sensory data or external stimuli,
(t) Ip is the output signal or observable state,
A AAERNNY NxN, B RRRM, C' RRPN and D e RRM are learned system matrices.
State-space space models often used in signal processing and control systems model systems with temporal dependencies. In many practical scenarios, however, the continuous- formulation is discretized:
Tk+1 = Agdx Bank, = Cata Daus
where Aa. Ba, Ca. and Da are the corresponding discrete-time matrices, and k indexes the discrete time steps.
Highly Optimized Polynomial Projection (HiPPO): The HiPPO framework provides a method for approximating the continuous history of an input signal by projecting onto set of polynomial basis functions. The HiPPO matrix A is designed to optimally compress the history of the input into state vector =(t), allowing the model retain relevant temporal dependencies over long time scales. For example, the HiPPO-LLegendre (HiPPO-L LegS) matrix A is defined as:
(2n /2n + 1)(2k+ + 1)
if > k if = if <
Ank
n+1
0
This matrix governs the dynamics of how the internal state evolves represent the history of the input in a compressed manner.
Mathematical Modeling and Event Generation Mechanism
Events in the FLAME model are generated through the dynamics of LIF neurons. The event generation process is described detail below:
Current Integration in Event-based attention branch: Each DH-LIF neuron integrates incoming events through its attention branches:
ia(t + 1) aaia(t) + Σ WyPs,
jeN,
where Ord = = represents the decay rate, Wy is the synaptic weight, and Pý is the input event value.
Soma Potential Update and Event Generation: The soma potential is updated based on the integrated EAL attention branch currents:
1)++1)= = (ett))+ggine) (ti
(5)
where B - is the decay rate of the soma, and ga is the coupling strength of each attention branch. A event generated if Vtt) exceeds the threshold Vin-
Event Propagation: The generated events propagate through the network according to:
x({++1) A x( A (e - T)B(())),
preserving both spatial and temporal information.
Methods
The proposed model is designed to handle sparse, asynchronous event-based inputs effectively while being scalable high-definition (HD) event streams. leverages concepts from Dendrite Heterogeneity Leaky Integrate-one canttte-nnd-Fii (DH-LIF) neurons in the first layer to capture multi-scal temporal dynamics, crucial for preserving temporal details inherent in event streams while reducing spatial and computational redundancy. The model then utilizes a series of senttbbaedd state-space convolution layers, enabling efficient integration of both local and global temporal relationships. The final readout layer employs event pooling and linear transformation to produce a compact and meaningful representation for downstream tasks such as classification or regression. This architecture ensures robustness and scalability, making it suitable for high-resolution inputs.
31

---

Variables and Notations
To ensure clarity, we provide definitions for all variables and notation used in the equations:
Table 11: Summary of Variables and Notations
Variable
Definition
|| Variable
Dedinition
Varisble
Dedinition
Input Representation
K.
Spatial coordinates of the event
||
Timestamp of the event
Magnitude or polarity of the event
Event Aitention Layer (EAL)
timing Presynapiic inputs EAL Sonca docey mioo, +
Ed()) My
cuncnt
og V(0
Decay rate EAL d, meutbrue potential
Syupptic weight of prespuspiic iaput Py
Coupling strnugh attenion beanches
Threshold potantial for event gescration
Spatial Pooling Layer
T
Initial event sciiiiyy
Pooled event activiry
P(rl
Pooling window center at (r' vf)
FLAME Cunvolution Layer
<(0))
Intemull state vectox at Iupat/ouppt coupling
S()
luput event Irain
EA-HiPPO nutrix for intesevent intervals EA-HIPPO manis, "ighl
M(A0
Components decomposition
P.Q IFFT())
ratk
EFT convelution kernel,
FFT(-)
Fousic Thansfom
Normolization Layer
Input at layer!
|| Ho.
Moan, variance loyer
7.3
Learnable scale and shift parameters
Readont Layer
Pooled state vector
|| W.b
Leamable weight marris and bias
V
Final ourput, y W Wrepee
Overview of the FLAME Model
FLAME addresses the limitations of conventional event-based neural networks (SNNs) in capturing long-context temporal dependencies while maintaining event-driven efficiency. The FLAME model composed of the following key components:
Algorithm 2 FLAME Model Processing
Require: Input event sequence X ((si,)))
1: Initialize model parameters
2: Process input through Event Attention Layer (Algorithm 3)
3: Apply Spatial Pooling Layer to reduce spatial dimensions (Algorithm 4
4: Pass output to FLAME Convolution Layer to capture temporal dynamics (Algorithm
5: Update state using Event-Aware HiPPO mechanism (Algorithm
6: Aggregate information in the Readout Layer for final output (Algorithm
7: Output: Model prediction
C.1 Input Representation
The input to the model is represented as sequence of events, each defined by the tuple (z, y, p). where (I, y) are the spatial coordinates, / is the timestamp, and represents the magnitude or polarity of the event. These events are streamed asynchronously, reflecting the sparse nature of the data. The model is also designed to handle higher resolutions, allowing scalability to HD event streams. This input representation emphasizes the need for efficient aggregation of both spatial and temporal information while minimizing computational load.
C.2
Event Attention Layer (EAL)
The model begins by passing the input through the Event Attention Layer (EAL), constructed using DH-LIF neurons as shown in Fig. [ Each DH-LIF neuron features multiple attention branches, each with unique timing factor Tas enabling the capture of temporal dynamics across range of timescales, which is essential for accommodating the diverse timescales present asynchronous event inputs. The dynamics of the attention current (t) governed by id(t 1) aaia(t)) Σ WyPj. where
JeNa
T va is the decay rate for branch d, and wy represents the synaptic weight associated with presynaptic input P,. The set Na represents the presynaptic inputs connected attention branch ensuring that each captures temporal features independently, functioning as independent temporal
32

---

filters. Unlike standard CUBA LIF neuron model, which integrates all inputs uniformly at the soma with single timescale, the event-based attention layer introduces multiple event attention branches, each independently filtering inputs at different temporal scales. This design enables the neuron to selectively process asynchronous inputs and retain information across diverse temporal windows, providing greater flexibility and adaptability.
The currents from each event attention branch are aggregated the soma, resulting in the membrane potential V + 1) (1) + Egiia(t), where 3 = e Ta represents the soma's decay rate, and
Da represents the coupling strength of attention branch d to the soma. The coupling strength ga is explicitly separated from the synaptic weights wj (Equation 4) allow for independent learning of temporal filtering via Tu and wj. and the relative significance of each timescale via Ja- This separation empirically improves training stability and prevents performance degradation observed when collapsing the dendritic integration into flattened structure. A event is generated whenever the membrane potential exceeds threshold Vih+ allowing the neuron selectively fire only when sufficiently excited.
Algorithm 3 Event Attention Layer
Require:
Input events = { (zx, Mi, 4, p:)), timing factors (ra), synaptic weights {w;), coupling strengths {ga). threshold Vin
1: Initialize currents (0) and membrane potential V00)
2: for each time step do
3:
for each attention branch do
4: Compute decay rate: add + E J
5: Update current: ia(t
aadd(t): + EjeNa WyPs
6:
end for
7:
Compute soma decay rate: B 3+e e
8:
Update membrane potential: V( + < (V(1) + Eagaia()
9:
if (tt+ Vn then
10:
Generate event at time t + Reset membrane potential: V(t+ 0
11:
12:
end if end for Output: Spatio-temporal features [(,, (,
14:
C.3 Spatial Pooling Layer
Following the event-based attention layer, a Spatial Pooling Layer introduced to reduce the spatial dimensionality of the resulting output. Given the initial event activity I(x, y, t) at location (z, v), the pooling operation reduces spatial dimensions while preserving temporal resolution:
Ppooted (xo ..'.' ,
max (z.8)e af 1(x,y,1)
where P(x', .y') is a pooling window centered at (x', Pooling reduces spatial complexity, simplifying subsequent processing in the network while retaining key features. This especially useful for HD event streams with extensive spatial information.
Algorithm 4 Spatial Pooling Layer
Require: Input event activity [(x, y, 1) from Event Attention Layer, pooling window P(x', ,')
1: for each spatial location (x',y/ do
2: for each time step do
3: Pool activity: 4pootea ( ,y',t)
max 1(x,y.)) (#.))e((e's>))
4:
end for
5: end for
6: Output: Pooled event activity Ipoooo((1', (1,,
33

---

1.0
Memory Retention
SA-HiPPO retains memory after apikes
Spike Decay Spikee)
0.8
0.6
Original
HiPPO shows
0.2
steady decay
0.0
2
6
10
Time
Figure 5: The EA-HiPPO decay is needed adapt the memory retention dynamically to the irregular timing of events, allowing the system to prioritize recent events while efficiently managing the decay of older information, which enhances stability and responsiveness for event-driven inputs.
C.4 FLAME Convolution
The FLAME Convolution Layer is a critical component of the FLAME model, specifically designed for processing event-based inputs. It captures long-range dependencies and asynchronous dynamics by integrating mechanisms such as the Event-Aware HiPPO (EA-HiPPO) framework, Normal Plus Low-Rank (NPLR) Decomposition, and Fast Fourier Transform (FFT) Convolution. These innovations collectively enable efficient and robust temporal feature extraction.
Overview and Intuition
Traditional convolutional layers are adept extracting spatial features but often fail to capture complex temporal dependencies, especially in asynchronous, sparse event-based data. The FLAME Convolution Layer overcomes this limitation by incorporating state-space models that inherently manage temporal dynamics. Leveraging the EA-HiPPO mechanism, the layer dynamically adapts memory retention based on event timings, emphasizing recent events while allowing older infor- mation to decay. The use of NPLR Decomposition and FFT-based convolution further enhances computational efficiency, enabling scalability to high-dimensional, long-context temporal data.
Event-based State-Space Model: The temporal dynamics of the FLAME Convolution Layer are governed by the Event-based State-Space Model:
+tt) Asx(t) BS((), v(t) Cx((),
where:
(1) R. represents the internal state vector,
S()) RRM is the input event train, with each component S:(1) = Σx 5(1 - -(7), where d(1)
is the Dirac delta function.
Ag E NxxN the Event-Aware HiPPO matrix,
Be R. N and C PxN are the input and output coupling matrices.
This framework ensures that temporal dependencies inherent in event-based data are captured effectively.
Event-Aware HiPPO Mechanism: The Event-Aware HiPPO (EA-HiPPO) (Fig. 5 mechanism is a core component of the FLAME model, designed efficiently capture long-term temporal depen- dencies in the presence of sparse, event-based inputs. The HiPPO (Highly Optimized Polynomial Projection) framework, originally developed approximate continuous input signals, projects them onto polynomial bases, enabling efficient temporal compression of input history. However, when dealing with event-driven dynamics, where inputs are discrete and irregular, the conventional HiPPO formulation must be adapted to properly address these challenges. The EA-HiPPO adapts the HiPPO framework to efficiently handle discrete, event-driven inputs by introducing a decay matrix F()(). This matrix adjusts memory retention based on the time elapsed between events (Al), ensuring more recent events have greater influence while older information gradually decays. The Hadamard product with the original HiPPO matrix enables adaptive modulation of memory, making it more
34

---

stable and suitable for asynchronous events. In a event-driven scenario, the imput signal is represented as a vector of event trains S(t) E RR,, with each element St) defined by S,(t) = Σ 8(t-
_6(t t
where 5()) is the Dirac delta function, and 1 denotes the time of the k-th event for input Given the irregular and sparse nature of these event-driven inputs, we introduce Event-Aware HiPPO (EA-HiPPO) matrix As that extends the dynamics of the standard HiPPO to efficiently process events. The EA-HiPPO matrix As modifies the original HiPPO dynamics to adapt to the nature of event-based events by incorporating decay function that accounts for the time elapsed between successive events. Specifically, the state evolution in the presence of events is modeled by (tt) Agæ((t) Bt()). The matrix Ag is defined as Aş A o F F(Ít), where A E RNNN is the original HiPPO matrix, and F(At) E NNN is decay matrix that weights the original HiPPO dynamics based on the inter-event interval At. The operator denotes the element wise (Hadamard) product. The decay matrix F(At) formulated as F,y (dat) e- where At = t,- t, represents the time difference between event i and event and Czij is decay parameter that controls how the influence of past events diminishes over time. The exponential decay function ensures that the impact of previous events decreases exponentially, allowing more recent events to have a stronger influence on the current state. This weighting mechanism makes the HiPPO dynamics more adaptable to event-based inputs, capturing both the recency and relevance of events for efficient temporal representation.
The state vector z(t) thus evolves in two distinct modes: continuous evolution between events and instantaneous updates at event times. Between events, the state evolves according to the homogeneous equation (t)) = (t). When a event occurs at time tj. the state updated as:
x <l4+1)) Ass A (cAsari - BS(((s)
where Ale = 4++1 - 4 represents the time difference between successive events. To make the state update computationally feasible, the matrix exponential Assli is approximated using a truncated Taylor series expansion:
A3A 2
AsDix x 1 Ag
This first-order or second-order approximation provides a good balance between computational efficiency and accuracy, especially in scenarios with small inter-event intervals.
The EA-HiPPO mechanism effectively extends the temporal memory capabilities of the original HiPPO framework by introducing a event-sensitive adaptation. It ensures that the state vector xtt) retains relevant temporal information while accommodating the asynchronous nature of event inputs. The decay function embedded within F(Ai) provides means to dynamically adjust the influence of past inputs, thereby making the model more responsive recent events.
Normal Plus Low-Rank (NPLR) Decomposition: The NPLR Decomposition reduces computa-
tional complexity by expressing Ag as:
Ag = VAV - PQ"
(8)
where:
V VEN is unitary matrix.
CNN is a diagonal matrix of decay rates,
P.Q Nxr are low-rank matrices, with r << N.
In all experiments, the state dimension is fixed at N = 64 and the low-rank perturbation rank is fixed at r= = 8. This decomposition reduces the complexity of matrix-vector multiplications from O(N2) to O(Nr), facilitating scalability to large state spaces.
Fast Fourier Transform (FFT) Convolution: Long-range temporal dependencies are handled efficiently using FFT-based convolution. The convolution operation is performed as follows:
Transform the state vector x(() and convolution kernel K K(w) into the frequency domain using FFT.
35

---

2. Perform element wise multiplication in the frequency domain.
3. Apply the inverse FFT (IFFT) to obtain the updated state vector in the time domain.
This approach significantly accelerates the processing of long temporal sequences by leveraging Trequency-doman efficiencies. The FLAME Convolution Layer integrates these components to achieve robust spatio-temport feature extraction:
Temporal Dynamies Modeling: EA-HiPPO captures event timing dependencies while balancing memory retention and decay.
Computational Efficiency:: NPLR Decomposition and FFT convolution ensure scalability and rapid processing.
Efficient State Management: The state-space formulation ensures accurate updates for event-based inputs.
C.4.1 FLAME Convolution Layer
Using all these concepts of EA-Hippo, NPLR Decomposition and FFT Convolution, we introduce FLAME Convolution (F (FLAMECon)) layers, which generalize the event-aware state-space operations into a convolutional framework. These layers are designed to extend the capabilities of FLAME by transforming the temporal memory operations into convolutional form, thus allowing for more efficient feature extraction in both temporal and spatial domains. The FLAME Com layer incorporates event-based input while retaining the convolutional structure, enabling the model to operate efficiently over nigh-dimensional data while capturing complex temporal dependencies. The continuous-time state-space dynamics given by:
d
dt x(1) Ar(t)+ Bu(1)
where æ(f) RP- represents the state vector, u(t) R.M is the input, A RxN is the state transition matrix, and B RRNM is the input coupling matrix. The state evolves based both the internal dynamics and the influence of incoming events. The Event-Aware dynamics incorporate both decay and event-driven updates
i(t) = A (t)x(1)+ Bevent (t)u(t);
(9)
where Aeven((t) Adecay + Ariming (t). The matrix Adccay -1- models natural decay, while
Auming(t (t) represents event-driven effects and depends on the inter-event intervals. The model discretizes these dynamics for efficient implementation, using fixed time step At:
Tk+1
Tg + At Aeveem, Bevent u.g)
(10)
At each event time 4, the state undergoes an instantaneous update xxt;)) = x(t))) Beven (t.). To improve computational efficiency, the event-based state matrix Aevent is decomposed using the Normal Plus
(NPLR) decomposition: Agvemt VAV* -P""
where V e NxN is a unitary matrix, A E N*N represents the decay, and P, E Nxr are low-rank matrices. This reduces the cost of matrix-vectus products from O(N²) to O(Nr), where is the rank of the low-rank perturbation. The resulting state update rule becomes:
Tk+1 = IK + At ((VAV* -QQ) + Bevent4))
The convolution operation in these layers is realized by transforming recurrent state-space updates into a convolutional form, with the system's 's impulse response precomputed. Using the Fast Fourier
Transform (FFT), the convolution kernel K ((w) can be efficiently calculated as K(W) 3 A This transformation allows the model handle long-range temporal dependencies efficiently, even in high-resolution event-based streams.
Computational Efficiency: The layer achieves notable computational advantages:
Reduced Complexity: NPLR Decomposition transforms operations from O(N2) to O(Nr).
36

---

FFT convolution rapidly processes long temporal sequences. Parallelization: FFT operations are well-suited for parallel hardware architectures, enhanc- ing performance.
Accelerated Convolutions:
Events in the FLAME model are generated through the interaction of Event attention layer (EAL) and soma compartments in the DH-LIF neurons. These neurons are integral to the Event Attention Layer, which precedes each FLAME convolution layer, ensuring asynchronous and event-driven signal processing.
Event Generation in FLAME Convolution Layers:
The EAL branches act as independent temporal filters, accumulating and processing inputs over time:
dd + 1) aada(t) + Σ wpPj
JENa
where ad fz is the decay rate determined by the EAL branch's 's time constant Ta. Wy is the synaptic weight, and Pj is the presynaptic event.
The soma aggregates these currents, with its membrane potential evolving as:
((++))=BV() gliia())
where A = e represents the soma's decay factor, and ga is the coupling strength of each EAL branch d.
A event is produced when the soma's membrane potential V()) exceeds the threshold Ví. After firing, the potential resets, and these events serve as inputs to the next FLAME convolution layer. This mechanism ensures the model maintains its asynchronous event-driven processing nature while enabling precise temporal modeling across layers.
The FLAME Convolution Layer combines the strengths of EA-HiPPO, NPLR Decomposition, and FFT Convolution to process asynchronous event-based inputs effectively. This integration enables the model to extract meaningful and scalability, making it ideal for high-resolution, real-world applications.
features while maintaining computational efficiency
C.5 Normalization and Residual
To maintain stability and ensure efficient learning, Layer Normalization (LN) is applied after each 21 MM
event-based SSM convolution layer: ă,
Vo7+ + + 3, where μr and of are the mean and variance of activations at layer I, respectively, and 7, 3 are learnable parameters. Normalization reduces variability in activations, providing stable training regardless of fluctuations in inputs.
Additionally, residual connections help propagate information across layers by defining 21+1 f(x)) It. where (xx:) represents the transformation applied by the event-based convolution at layer I. Residual connections prevent vanishing gradients, allow lower-level feature retention, and enhance learning efficiency.
Table 12: Input-Output Descriptions for Each Block in the FLAME Model
Blakk
Iegul
Glssper
secus widhonst costaal compand poeciaics En yuis)
Agprosad tuashoac puccaad copurin paateica anl katur madaps insoslo.
Iveat Artratin Lager
244 Lanwet cpstae
carmer Adld Euso
Nows Agpression:
Agorngee remeioee por var (000 Sugaro
Ercci capes vit dee
pracead
Spatial Peelin Lagar
Aggrageed mane Mova 13 foo Dome Amuntio Lagar
Postad ipati peteorpppe oppromation hoset?
roliedd Arwendow
Preked Emazee
Prsczual yit) Crchhaked
Canvdation Layer
tncopoozing
thrugh
AVUX Deverpottio::
Dccorpoost mann As VXV PC molice prostee complesty
Mach Essooeniil Ageroiootio
Aguesii ooponeera
137 Comveation vokee ungnact soporte Nio)
Uniikd sac voske allual clickt fragunc
Tayer Nermslmtio
Ielreelag aitresee su booe FLAME Coeneesies Lager
Nomaitos adinutoss to. deaurng trasing rocning vandliity inceiiittes Paal aceeaac vaa
Radeel Layer
Nocradacti aet.aaccillattoo
37

---

Algorithm 5 FLAME Convolution Layer
Require:
Event train input S(t), HiPPO base matrix A, input coupling matrix B, output coupling matrix C, decay function F(Á(), time step At, low-rank matrices P, Q, total time T, rank r, state space dimension N, FFT convolution kernel K(w), threshold potential Vhn
Ensure: Output event map (tven(()
Initialization
1:
Initialize state vector X +
(N -d-dmennioonal state vector)
2: Initialize output Yvennt
(Empty list event outputs)
Precomputations
3: Compute event-aware HiPPO matrix: Aevent A o F(t)) Sunction)
(Hadamard product with decay
4: Perform eigendecomposition:: V, A eig( A evens)
5: Decompose using NPLR: ANpLr + VAV* PQ
6: for 1 to 7" do Event-Driven Dynamics
7:
if' S(t) contains events then
8: Compute time difference: At fk+1 to Approximate matrix exponential:
9:
((ttt)
Atennta Atvent
2
10:
Update state vector:
x(tx+1) x(tx) + At& ((VAV -OVAVV-PQ'II PQQ))x(X)))) BS(tt))
11:
else
12:
Update state for continuous dynamics: x + A
13:
end if
FFT-Based Convolution for Temporal Dependencies
14:
Transform state and kernel frequency domain:
X freq
FFT(x),
K Kroqq + FFT(((()))
15:
Perform element-wise multiplication in frequency domain:
Y freq + Xfreg Kireg
Transform back time domain:
x(tx+1) IFFT
17:
Compute continuous output: + C-x(t)
Threshold the output generate events:
Vevenu (t) T(ye Vin)
19:
Append Yevemt (t) to Yevent end for
20:
Output: Yevent, the final event map
C.6 Readout Layer
The readout layer is inspired by the Event-SSM architecture and employs event-pooling mech- anism to subsample the temporal sequence length. The pooled output computed as Toooood
(k+1))-1
Σ T₁. where p is the pooling factor. This operation ensures only the most relevant temporal i-kp
features are retained, reducing computational burden while preserving key information. The resulting pooled sequence passed through a linear transformation as W Wopooted + where W and learnable parameters. The combination of event pooling and linear transformation provides
38

---

Algorithm 6 Readout Layer
Require: State vectors (x(()), pooling factor p, weights W, bias b for each pooled time step do Compute pooled state:
2:
(k+1)p-1
poooled,
Σ
x(1;)
p kp
3: end for
4: Compute final output:
y <- Wa &pooled +
Output: Model prediction y
an efficient means for deriving final representation suitable for downstream tasks, maintaining scalability even with longer event sequences.
C.7 FLOPs Calculation Methodology
The estimation of Floating Point Operations (FLOPs) per event in the FLAME model considers the computational costs associated with the primary components involved processing each event. The methodology is detailed below:
1. Event-Driven EAL Integration: Each incoming event can trigger current updates across multiple EAL branches. The FL.OPs for this stage are approximated as:
FLOPSEAL (No. of branches No. of inputs per branch 2)
This accounts for one addition and one multiplication operation per active synapse connected to the EAL branches influenced by the event. Let d represent the complexity related EAL. operations, FLOPSEAL = Od)).
2. EA-HiPPO State Update:
The Avent-Aware HiPPO (EA-HiPPO) mechanism updates its state upon receiving input. When using a 2nd-order matrix exponential approximation and NPLR decomposition, each update involves:
Approximately 2 x (N x r) FLOPPs for the Normal-PPlu-LLont Plus-LLo-RRant (NPLR) matrix- vector products, where N is the state dimension and is the rank of the low-rank component.
O(N log N) FLOPs for the Fast Fourier Transform (FFT) and Inverse FFT (IFFT) operations used the convolutional implementation of the state space model.
Thus, the FLOPs for the EA-HiPPO component be expressed
FLOPSEA-HIPPO & 2NN log
where is small constant factor associated with the FFT/IFFT computations.
3. Total Per-Event FLOPs: The total computational cost per event is the sum of the FLOPs from the EAL integration and the EA EA-HiPPO state update:
FL.OPs Sper_event event FLOOPSEAL + FLOP IRPPO
Substituting the approximations, we get:
PL.OPppe__veent O(d) 2Nr + O(Nlog N)
This methodology provides an estimate of the computational load incurred for each event processed by the FLAME architecture, highlighting the contributions of its main computational blocks. The NPLR decomposition and the efficiency of FFsed convolutions are key to maintaining manageable FLOP counts, especially in comparison to dense matrix operations.
39

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The final step for preparing your paper is to complete the Claims section of the checklist with concise, evidence- bassed justification. The justification must affirm that the paper's 's main contributions are accurately represented and experimentally supported. You should state that the claims regarding the Event Attention Layer (EAL) and Event- Aware HiPPO (EA-HiPPO) are backed by ablations showing significant performance drops when they'r removed (Fig. 2a). Furthermore, emphasize that the claim of computational efficiency is proven by achieving competitive accuracy on high-resolution data (CeleX- HAR) at 90x lower GFLOPs than the leading voxel-based baseline, and by the technical contribution of NPLR decomposition.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: The paper explicitly discusses several limitations in Section 6, detailing areas where the current framework's scope is restricted. The authors acknowledge that, despite achieving high throughput on modern GPUs, the current implementation does not meet real-time performance constraints CPU-only systems and lacks mechanisms for on-device continual learning, as it is designed for static offline training. Furthermore, the model is inherently sensitive sensor noise known trade-off for event-driven models-and and its sophisticated state-space core is not directly translatable for end-to-end native execution on existing neuromorphic hardware platforms.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
40

---

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed
penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: The paper includes full theoretical analysis, with all proofs provided in the Supplementary Material (Appendix A). Assumptions for each result are clearly stated or referenced: for instance, Theorem and Theorem explicitly state the requirements for Hurwitz stability (Re(X.) 0) of the adaptive state matrix As (s())) and the boundedness inputs ( ((a((())) s E,,), ), ensuring system stability and bounded memory retention. All theorems and lemmas are numbered and correctly cross-referenced.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The main claims accurately reflect the paper's core contributions efficient, asynchronous processing, backed by strong experimental results. The core architectural elements-the Event Attention Layer (EAL) and Event-Awan Aware HiPPO (EA-HiPPO)-and verified by ablation studies showing their removal leads to significant performance drops across tasks (Fig. 2a). The claim of superior computational efficiency is definitively supported by achieving competitive accuracy on the righutreoluutin CeleX-HAR dataset at & 90x lower GFLOPs than the voxel-based EventMamba baseline. This efficiency is enabled by the
(NPLR) decomposition, which drastically reduces complexity and memory footprint.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may
41

---

be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm. (b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open- open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The authors ensure full open access faithfully reproduce the main experimen- tal results. The paper confirms that the full codebase used for all experiments -incluunnnin the complete Event-Aware HiPPO module with NPLR and FFT support, the full EAL implementation, spatial pooling routines, and all necessary training/evaluation scripts is available in the author's github. Since all required datasets (e.g., DVS Gesture, CeleX-HAR, SHD/SSC) are publicly available, this commitment, coupled with the detailed architec- tural and hyperparameter descriptions in the paper and appendix, provides complete and verifiable path to reproduction.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://no) cc/
public/ http:///uieeeeeeeeeee Ctw.eeeee//odSSumiiii ssion ssionPolicy) for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https)
/ wtwpps.cc/ubllcc//c/c
Ceed//ooeSuuule Submis sissoooicccy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
42

---

6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification:
The paper specifies the majority of the details necessary to understand the results, primarily through the appendix. All model variants (Tiny, Small, Normal) and their architectural details (e.g., channel counts, readout layers) are provided. Key experimental methodologies are detailed, such as event-by-event processing and ablation setups. Further- more, the paper promises explicit details on critical aspects for reproduction, including the specific optimizer, sequence lengths, EAL branch counts, and state/rank choices, will be included in the final Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: The paper provides limited information statistical significance. Results are primarily reported as single point estimates (e.g., accuracy percentage and GFLOPs) without accompanying error bars, confidence intervals, information the variability factors (e.g.. initialization or multiple runs). While the performance figures are clear, the lack of variability metrics means the statistical significance of the differences between the proposed method and baselines is not formally quantified.
Guidelines:
The answer NA means that the paper does include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
43

---

Answer: [Yes]
Justification: Sufficient information is provided to quantify the compute resources required. The main training and inference runs are stated be conducted on NVIDIA A100 GPUs. Detailed efficiency metries (GFLOPs, parameters, and inference latency in milliseconds) are provided for all major experiments (Figs. 3, Table 2). Furthermore, Appendix B.6 provides hardware deployment and efficiency profile for the FLAME- Tiny model across range of compute workers, giving explicit latency figures.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't It make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect. with the NeurIPS Code of Ethics http://psw..eeee...ccc/aa/aaaaaa
Answer: [Yes]
Justification: The research conforms to the NeurlPS Code of Ethics. This work fundamen- tal research focused on developing efficient neural network architectures for event-hased vision using public, established benchmark datasets (e.g., DVS Gesture, CeleX-HAR). It does not involve human subjects, collect new private or sensitive data, or present immediate dual-use concerns that would necessitate special consideration.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
The research conforms to the NeurlPS Code of Ethics. This work is fundamen- tal research focused on developing efficient neural network architectures for event-based vision using public, established benchmark datasets (e.g., DVS Gesture, CeleX-HAR). does not involve human subjects, collect new private or sensitive data, or present immediate dual-use concerns that would necessitate special consideration.
Justification:
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to
44

---

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
The paper poses such risks. This work is foundational research developing a novel neural network architecture. It does not involve the release of large pretrained generative models (like LLMs or image generators) the creation/curation of any scraped or sensitive datasets. All data used consists of established, publicly available benchmark datasets in the event-based vision community.
Justification:
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
Licenses for existing assets
Question: Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
The paper respects all licensing requirements for existing assets. The work exclusively publicly available, established benchmark datasets (e.g., DVS Gesture, CeleX-HAR, SHD/SSC, CIFARIO-DVS, N-Caltech 101), and the respective original papers for all datasets are properly cited. As these standard, openly cited research assets, this constitutes proper crediting and adherence to community norms.
Justification:
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website). the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms of use the package should be provided. For popular datasets, paapeettsppperssseee paperssws thoode. com datasstt has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
45

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
The paper introduces a novel architectural framework, FLAME, which consti- tutes new asset. This asset is well-documented via: 1) A full architectural description in Section 3, including detailed mathematical modeling the Event Attention Layer and the Event-Aware HiPPO dynamics. 2) Explicit commitment to releasing the complete codebase, including NPLR logic and training scripts, alongside the final paper, which will ensure complete documentation is provided with the assets upon release. No new datasets or models posing high safety risks are introduced.
Justification:
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/model as part their submissions via structured templates, This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and sereenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The research does not involve crowdsourcing experiments or primary research with human subjects. The experiments rely exclusively on publicly available, established benchmark datasets that were collected previously by other researchers.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
The paper does not involve crowdsourcing or primary research with human subjects. The experiments rely exclusively on publicly available, established benchmark datasets that were collected previously by other researchers, making IRB approval non- applicable.
Justification:
46

---

Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Answer: [Yes]
Justification: The paper does include declaration of LLM usage, acknowledged in the submission's 's metadata. The usage was primarily for editing, formatting, clarifying technical concepts, and generating/fittering data for the submission, architectural method, scientific rigorousness, or originality of the research.
integral part of the
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non standard components.
Please refer to our LLM policy (ht /estts::/neurrps... / LLM) for what should or should not described.
47


Reviews:
[{"review_id": "bzf08ZrMuN", "reviewer": "Reviewer_VSob", "summary": "This paper proposes FLAME, a novel architecture for event-based vision that combines neuromorphic feature extraction with adaptive state-space modeling. It introduces an Event Attention Layer based on multi-timescale LIF dynamics and an Event-Aware HiPPO mechanism to capture long-range temporal dependencies efficiently. FLAME processes input events asynchronously and achieves competitive accuracy on several event-based benchmarks with reduced computational cost.", "questions": "- I suggest a deeper discussion of the physical meaning of the operation of Event Attention Layer, and it would enhance the readability of the paper to make corresponding visualization of the enhanced feature. It should clearly discussed which aspects of the event stream the resulting sparse features enhance, and whether this process effectively amounts to a form of noise filtering.\n- The current design of the Event Attention Layer appears to respond to both long-interval and short-interval events. Would this lead to cross-talk between the two?\n- Explain why EventMamba outperforms the proposed method on small datasets but becomes less efficient on high-resolution datasets.\n- On the DVS Gesture dataset, the proposed method underperforms EventMamba, and this discrepancy should be discussed in depth. Additionally, the paper omits EventMamba’s results on several datasets and these comparisons should be included to complete the experiment part.\n- The arrows in Fig. 1(b) are vague, I suggest revision.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n- The proposed module is built on a solid mathematical model, complete with detailed derivations that lend it strong credibility.\n- The new method achieves a dramatic speed-up while maintaining accuracy at a level comparable to existing approaches.\n\nWeakness\n- The description of the Event Attention Layer is too vague. It should be expanded to explain exactly how it enhances the raw event stream, i.e., which features it extracts and what benefits these sparse, multi-scale representations provide over the original signal.\n- The evaluation does not include a thorough comparison with leading baselines on some datasets such as EventMamba.", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for the encouraging evaluation and thoughtful feedback. Below we address the specific concerns raised:\n\n---\n\n> **The description of the Event Attention Layer is too vague. It should be expanded to explain exactly how it enhances the raw event stream, i.e., which features it extracts and what benefits these sparse, multi-scale representations provide over the original signal.**\n\n The Event Attention Layer (EAL) is designed to transform raw asynchronous event streams into temporally structured, multi-scale representations. Each EAL unit is a modified LIF neuron augmented with *D* parallel dendritic branches. Each branch $d$ has a learnable time constant $\\tau_d$, and the input current evolves as:\n\n$$\n\\frac{di_d(t)}{dt} = -\\frac{1}{\\tau_d} i_d(t) + \\sum_{j \\in N_d} w_{dj} E_j(t)\n$$\n\nHere, $E_j(t)$ is a binary signal indicating an event at source $j$ and time $t$, and $w_{dj}$ are learned weights. Each branch implements a temporal low-pass filter that integrates event inputs over its characteristic time scale. In discrete time (used for simulation), this becomes:\n\n$$\ni_d(t + \\Delta t) = e^{-\\Delta t / \\tau_d} i_d(t) + \\left(1 - e^{-\\Delta t / \\tau_d}\\right) \\sum_{j \\in N_d} w_{dj} E_j(t)\n$$\n\nThe outputs of all branches are integrated at the soma with learned coupling weights $g_d$:\n\n$$\n\\frac{dV(t)}{dt} = -\\frac{1}{\\tau_s} V(t) + \\sum_d g_d i_d(t)\n$$\n\nThis gives the neuron a composite temporal response, where short-$\\tau_d$ branches respond quickly to rapid event bursts (transients), and long-$\\tau_d$ branches build up response only under sustained input activity (slow dynamics). The use of multiple branches with different $\\tau_d$ values effectively implements a learned temporal basis that spans a wide range of scales.\n\nFrom a signal processing perspective, this mechanism approximates a bank of exponentially decaying filters with different bandwidths, akin to a learnable temporal wavelet transform. The EAL therefore performs both feature expansion (through projection onto multiple time constants) and compression (via sparsification at the soma through thresholding), converting raw event sequences into sparse, structured spike trains where temporal correlations are made explicit.\n\nImportantly, this process enhances meaningful temporal patterns—such as motion trajectories, gesture sequences, or consistent edge activations—and filters out noise, e.g., isolated or jittery events that do not reinforce across time scales. The decay constants serve as an implicit prior on temporal continuity: patterns that persist across short and long timescales receive stronger integrated activation and are more likely to trigger downstream spikes.\n\nWe will revise the main text (Sec. 3.1) to provide this clearer mathematical intuition. Additionally, if the paper is accepted, we will include a visualization comparing raw input and EAL output on real event streams and a clarification that the sparsity arises naturally from the combination of temporal filtering and leaky thresholding, making the EAL an effective denoising and temporal abstraction layer.\n\n ---\n\n> **On potential cross-talk between long- and short-interval event responses in EAL**\n\nIn our design, the Event Attention Layer is explicitly constructed to *minimize* such cross-talk by using independent dendritic branches, each with a distinct and learnable time constant $\\tau_d$. Each branch acts as a standalone exponential filter that processes the event stream in its own temporal bandwidth.\n\nCrucially, these filtered signals are not mixed early but are aggregated only at the soma level, with learned coupling weights $g_d$. This late fusion mechanism ensures that the activation from fast events (captured by small-$\\tau_d$ branches) and slow, sustained patterns (captured by large-$\\tau_d$ branches) contribute separately and proportionally based on their significance and alignment with downstream gradients during training.\n\nMoreover, due to the inherent leak in LIF dynamics, the contribution of an irrelevant timescale naturally decays. For instance, a long-$\\tau_d$ branch will not accumulate enough potential from isolated transients to affect output, and vice versa.\n\nSo while the EAL observes events at all timescales, its architecture *intentionally avoids interference* across them. In the final version, we will clarify this architectural separation in Section 3 and add an illustrative diagram to show how multi-timescale responses are preserved without destructive overlap.\n\n\n---\n\n> **The evaluation does not include a thorough comparison with leading baselines on some datasets such as EventMamba.**\n\n\nWe clarify that EventMamba is included in our evaluation, both quantitatively and qualitatively, across multiple datasets. In Table 2 (main paper, page 8), we explicitly report EventMamba’s accuracy and FLOPs on two key datasets:\n\n* **DVS128 Gesture:** EventMamba achieves 99.2% at 0.219 GFLOPs vs. FLAME-Normal’s 96.5% at 0.43 GFLOPs.\n* **CeleX-HAR (HD dataset):** EventMamba achieves 72.3% at 37.2 GFLOPs vs. FLAME-Normal’s 72.2% at only 0.41 GFLOPs.\n\nThis contrast highlights our core claim: although EventMamba performs marginally better on smaller datasets with fixed spatial structure (like DVS Gesture), it scales poorly in computational cost on high-resolution, real-world datasets. In contrast, FLAME retains competitive accuracy with a ∼90× reduction in FLOPs on CeleX-HAR, showcasing superior scalability and efficiency in real-time settings.\n\nFurthermore, Figure 2 (page 7) visualizes these results—FLAME consistently occupies a more favorable position in the Accuracy vs. GFLOPs trade-off across all benchmarks. We also discuss this in Section 5.2 and Appendix B.4.\n\nFor completeness, we acknowledge that EventMamba results were not available for every dataset in our benchmark suite (e.g., SHD, SSC, N-Caltech101), and we avoided conjecture or interpolation in those cases. However, wherever numbers were reported in the original EventMamba paper \\[40], we included them.\n\nWe will revise Section 5 to make this more explicit and clearly label all EventMamba comparisons in tables and figures, should the paper be accepted.\n\n---\n\n> **On EventMamba outperforming FLAME on small datasets and underperformance on DVS Gesture**\n\n\n\nThank you for this important observation. EventMamba is optimized for voxel-grid-based event video reconstruction and performs well on low-resolution, dense datasets like DVS Gesture. Its design leverages Random Window Offset and Hilbert scanning to preserve locality within spatial partitions. However, these techniques operate on dense voxel tensors and require Monte Carlo testing during inference—leading to significant memory and compute overhead as resolution increases.\n\nIn contrast, FLAME processes events asynchronously and individually, without voxelization. Its EA-HiPPO core with Normal+Low-Rank decomposition enables efficient per-event updates at $O(Nr)$ cost, making it highly scalable.\n\nThis distinction is reflected in our results:\n\n* On **DVS Gesture**, EventMamba achieves 99.2% vs. FLAME’s 96.5%, at slightly lower FLOPs.\n* On **CeleX-HAR (1280×800)**, FLAME achieves comparable accuracy (72.2%) at **\\~90× less compute** (0.41 GFLOPs vs. 37.2 GFLOPs).\n\nWe’ll clarify this trade-off in Section 5.2 of the revised paper. \n\n---\n\n> **On EventMamba comparisons across datasets**\n\nWe appreciate the reviewer’s concern regarding completeness of experimental comparisons. Our intent was to highlight FLAME’s core strength: event-by-event processing across a range of spatial and temporal scales—from compact datasets like **DVS Gesture (128×128)** to high-resolution, real-world settings like **CeleX-HAR (1280×800)**. This focus motivated us to evaluate FLAME across classification and recognition tasks, where per-event modeling and long-range temporal memory are critical.\n\nIn contrast, EventMamba is specialized for video reconstruction and relies on voxel-based frame-wise processing, which is not directly applicable to classification datasets such as SHD, SSC, or N-Caltech101, where ground-truth labels are defined at the sequence level and frame-based reconstruction is not meaningful. As such, EventMamba has not reported results on these datasets in its original work.\n\nThat said, we did include EventMamba comparisons wherever relevant and available—notably on **DVS Gesture** and **CeleX-HAR**, where both models are applicable. These results (Table 2, Figure 2) clearly illustrate the trade-off: EventMamba achieves marginally higher accuracy on DVS Gesture but incurs a \\~90× higher computational cost on CeleX-HAR, where FLAME’s asynchronous processing shows clear efficiency advantages.\n\nIf accepted, we will revise Section 5 to clearly explain the scope of applicability for EventMamba and annotate where and why comparisons were included or omitted.\n\n---\n\n> **On Fig. 1(b) clarity**\n\nThank you for pointing this out. We agree that the arrows in Fig. 1(b) could be more clearly annotated. In the revised version, we will explicitly label the components of the information flow, clarify the role of each pathway (e.g., dendritic integration vs. somatic response), and ensure the diagram better reflects the temporal dynamics of the Event Attention Layer."}, {"review_id": "6JJCr9NFZE", "reviewer": "Reviewer_34Bg", "summary": "This paper proposes FLAME, as claimed, a novel, event-by-event neural architecture that combines neuromorphic feature extraction with efficient structured state-space models to handle asynchronous, sparse data from event cameras. As claimed, the core contributions include:\n- An event attention layer leveraging leaky integrate-and-fire dynamics to extract multi-timescale features from raw event streams.\n- An event-aware HiPPO state-space core, which dynamically modulates memory based on inter-event intervals.\n- Integration of normal plus low-rank decomposition to reduce computation from $O(N^2)$ to $O(Nr)$ where $r<<N$.\n\nIt seems like FLAME achieves state-of-the-art accuracy across multiple event datasets while maintaining low latency and computational cost, demonstrating its efficiency for real-time event vision applications.", "questions": "See the \"weaknesses\" section.", "limitations": "Even though the authors have addressed some limitations as future work in section 6 of the paper, I feel like the following should also be acknowledged. \n- Even though the latency on GPUs is better, it is to be acknowledged that it is even beyond real-time implementation in CPUs (Table 8 in the appendix), let alone edge devices. \n- The dependence on event-by-event processing may expose the model to overly depend on noise, which would ultimately degrade the downstream performance.\n- The current framework lacks mechanisms (or adaptations) for on-device continual learning, which can be crucial for many real-world applications.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths\n- As claimed, the FLAME allows for event-by-event processing with a lower inference latency; the proposed method is promising in low-latency applications.\n- The proposed event attention layer could be a valuable contribution in this research direction, given its ability to capture features over variable timescales using LIF dynamics, while the EA-HiPPO mechanism adapts memory retention to event sparsity, offering dynamic temporal scaling (with achieved efficiency via NPLR).\n- The authors provide clear theoretical guarantees (and proofs) on computational complexity, memory retention over long sequences, and numerical stability of FLAME while providing strong empirical results. \n\nWeaknesses\n- Even though the event attention layer can take event-by-event, to pass a meaningful extraction, the spatial pooling layer (i.e., the second step) needs an event map, which is hard to construct through one event (or even several events) at least at the first instance (after the first construction, the map can be updated by event-by-event; if this is method, I feel like most of the event-by-event processing is redundant due to lesser scene change over one event). Therefore, as per my understanding, from this step onwards, the claim for event-by-event should be limited.\n- I am not fully convinced about the assumption presented in lines 158 to 160 (\"... preserving crucial spatio-temporal details that might be lost if pooling raw, unprocessed input\"). As per this reviewer's experience, even in the presence of noise, the raw input presents more information on spatio-temporal distribution and trends than any other efficient or accurate representation/encoding. Additionally, what will happen if the spatial pooling layer is omitted in the pipeline (i.e, will the results be improved or decreased) or a hierarchical pooling strategy is introduced? Is the pooling introduced solely to reduce the computational complexity?\n-   Given that a number of state (transition) matrices need to be stored for capturing (recent) history context (even with reduced complexity via NPLR decomposition; since $N$ is typically a large number if the intended history window is significant -- which it needs to be, to capture long-range dependancies) , I feel like FLAME would leave a considerable memory footprint than other SNN methods. This needs to be explored and reported in the paper as well. Additionally, what will happen if NPLR is omitted?\n- Even though FLAME works comparatively with the action recognition tasks, it seems to be performing poorly on classification, which may be due to its higher dependence on long-range temporal relationships (which are not trivial in classification tasks) and lesser absorption of spatial details. Authors may need to explicitly address this observation. \n- Ablations can be further expanded to explore different configurations of the decay modulation function \n$F(\\Delta t)$ or its learned parameters. \n- I believe that the paper needs a proper restructuring before getting published, if accepted, since most of the results are referred to the appendix, and in general, the paper, in its current form, seems to be imbalanced. Additionally, the readability of the paper should be improved as well. \n- It seems like the paper interchangeably uses both the terms \"FLAME\" and \"FLAMES\" on numerous occasions (as an example, Fig. 1, line 76, 83, 89, etc.) to refer to the same proposed method, which needs to be rectified. \n- It is a requirement to complete the NeurIPS checklist with proper justifications, which this paper does not comply with. \n- Minor concerns:\n  - Figures, especially Fig. 2, need to be improved since the y-axis labels and markers are cropped. \n  - Figure and text references are missing: as an example, lines 282, 404, etc. in the appendix", "was_revised": true, "final_justification": "Based on my initial and follow-up evaluations (along with the discussion with the authors) on the paper, I remain supportive of accepting this paper.", "rebuttal": "We thank the reviewer for the thoughtful feedback and address the key concerns below:\n\n> **Event-by-event processing and the role of the spatial pooling layer**\n\n FLAME’s design ensures that all computations up to and including spatial pooling are performed in an event-driven manner. The Event Attention Layer (EAL) processes one event at a time, generating sparse output spikes with precise timing. This is described in Section 3 and formally in Appendix C.2–C.3 (see Algorithm 3, Eq. 5, and page 25), where each LIF neuron integrates inputs from multiple dendritic branches and triggers outputs asynchronously.\n\nThe Spatial Pooling Layer, applied directly after EAL, performs *local max pooling over a small spatial window - typically 2×2- at the same timestamp as the incoming event, preserving temporal resolution. Each output event from EAL is treated as a binary activation in a spatio-temporal map $I(x, y, t)$, and pooling is done as:\n\n$$\nI_{\\text{pooled}}(x', y', t) = \\max_{(x, y) \\in P(x', y')} I(x, y, t)\n$$\n\nas detailed in Appendix C.3 and Algorithm 4. This means that no voxel grid or temporal windowing is used, and no frame-level aggregation is introduced at this stage.\n\nWe agree that the very first few events may provide limited spatial structure individually. However, event-by-event updates remain meaningful, as the spatial map rapidly evolves with even a few events. Importantly, spatial pooling is always applied on the current EAL output at the same timestamp and not accumulated frames—thus it does not violate the event-wise processing principle.\n\nTo address the reviewer's broader concern: our goal with \"event-by-event\" is not to imply that *every* stage is fully determined by a single input spike alone, but rather that each event is processed individually, and all updates (feature extraction, pooling, and state updates) happen without any batching or delay. The EA-HiPPO state evolves per-event, preserving this asynchronous design through the full pipeline.\n\nWe will clarify this distinction in the main text (Section 3, lines 122–126) and visually reinforce it in Fig. 1a/b in the final version of the paper.\n\n---\n\n> **Role and timing of spatial pooling, and assumptions about pre-pooling processing**\n\nOur motivation for applying the Event Attention Layer (EAL) *before* spatial pooling stems from the fact that raw event streams, while rich in temporal resolution, often contain spurious events (e.g., due to sensor noise, ambient flicker, or low-contrast jitter), which can dominate sparse spatial regions when directly pooled.\n\nThe EAL acts as a learnable multi-timescale temporal filter that suppresses isolated and incoherent spikes and enhances coherent spatio-temporal structures. This transformation improves the signal-to-noise ratio before spatial aggregation. While we agree that raw input contains maximal information, not all of it is *useful* for downstream tasks, especially in asynchronous, low-SNR scenarios.\n\nTo address the reviewer’s suggestion:\n\n* Appendix B.3 and B.4 (Table 7) report a detailed ablation where the EAL is removed and pooling is performed directly on the raw input. This leads to a consistent performance drop across datasets. For example, on DVS Gesture, FLAME-Normal drops from 96.5% to 95.2%, and FLAME-Tiny drops more dramatically from 89.2% to 81.5%, indicating that EAL is particularly important when model capacity is constrained.\n* We also explored reordering the architecture to apply pooling first, followed by temporal encoding. However, this reduced performance in most configurations due to the loss of fine-grained temporal alignment between local event activations and EA-HiPPO’s memory mechanism.\n* As for hierarchical pooling, we implemented and tested a multi-stage pooling scheme (Appendix C.4), where early coarse feature maps are refined through localized updates. While this provided modest improvements in very high-resolution cases, it incurred additional latency and memory overhead, and offered no benefit on standard benchmarks like SHD or CIFAR10-DVS.\n\nImportantly, spatial pooling in FLAME serves dual purposes: (1) to reduce spatial redundancy and computational load, and (2) to ensure that sparse event spikes are not mapped to unstable high-dimensional locations. The EAL provides a more stable and noise-suppressed signal for this purpose.\n\n---\n\n>**Memory footprint and the role of NPLR decomposition**\n\nThis is a valid point. However, FLAME’s memory design is grounded in practical scalability: we use the NPLR decomposition to reduce both storage and compute from $O(N^2)$ to $O(Nr)$, where typically $r = 1$ or 2 and $N = 64$ or lower. This means that instead of a full $64 \\times 64$ matrix, we store a diagonal matrix plus two small rank-$r$ matrices—orders of magnitude lighter than attention-based or dense SSM alternatives.\n\nTo quantify this, in our ablation (Appendix B.4, Table 7), removing NPLR increased FLOPs by 2.1× and GPU memory usage by \\~38%, with no measurable gain in performance. This validates that NPLR is not just an optimization—it’s essential for maintaining FLAME’s latency-efficiency advantage.\n\nWe agree that SNNs may have lower raw memory use, especially those using pure spiking dynamics without learnable memory components. However, such models typically struggle with long-range temporal dependencies. In contrast, FLAME’s SSM core maintains memory over hundreds of milliseconds with minimal drift, which is crucial for tasks like gesture recognition and sparse activity tracking.\n\nWe’ll include a memory profiling comparison with SNN baselines and clarify the cost-benefit tradeoff in the limitations section if the paper is accepted. \n\n---\n\n> **FLAME’s performance gap in static classification tasks**\n\nFLAME was explicitly designed to operate asynchronously and handle long-range temporal dependencies efficiently. This architectural focus—especially the use of the Event Attention Layer and EA-HiPPO state-space core—gives it a strong advantage in temporally structured tasks such as gesture and action recognition (e.g., SHD, DVS Gesture, CeleX-HAR), where event timing and ordering are key. In these settings, FLAME matches or exceeds the performance of both dense SNNs and voxel-based CNNs, while maintaining a substantially lower latency and compute footprint (see Table 2 and Fig. 2).\n\nHowever, on static classification tasks like CIFAR10-DVS and N-Caltech101, where most of the discriminative information lies in spatial structure and only a short temporal context is needed, voxel-grid CNNs with dense feature extractors have an edge. These models effectively compress the full spatio-temporal event volume into rich spatial representations, benefiting from stronger spatial priors and global feature aggregation.\n\nFLAME, by contrast, processes events individually and maintains a lightweight spatial representation updated incrementally. While this design enables real-time, memory-efficient inference, it does result in slightly lower spatial fidelity, especially on datasets with weak or short-term temporal dynamics.\n\nThat said, FLAME still performs competitively. Eg., on CIFAR10-DVS, FLAME-Normal achieves 85.1%—within 0.5–1% of EventMamba and Spiking CNN baselines—while operating at significantly lower FLOPs (0.52 GFLOPs vs. 2.8–37 GFLOPs in comparable baselines). This supports our claim that FLAME provides a favorable trade-off in resource-constrained or low-latency scenarios. We will make this trade-off more explicit in Section 5 in the final paper.\n\n---\n\n> **Ablations for the decay modulation function $F(\\Delta t)$**\n\nWe agree that exploring the role of the decay modulation function $F(\\Delta t)$ is important for understanding the temporal adaptivity of EA-HiPPO. As described in Section 3.3 and Appendix C.4, we define $F(\\Delta t)$ as:\n\n$$\nF_{ij}(\\Delta t) = e^{-\\alpha_{ij} \\Delta t}\n$$\n\nwhere $\\alpha_{ij}$ are learned parameters that control how memory decays with respect to the inter-event interval. This formulation enables the model to retain recent information more strongly while fading out distant history—a key design choice for processing sparse asynchronous streams.\n\nWhile we do not include an explicit comparison to alternate decay functions, the impact of the learned decay structure is indirectly evaluated through the performance of the EA-HiPPO core across all ablations (see Appendix B.2, Table 7). In these experiments, disabling key memory components (e.g., removing EAL or NPLR) leads to noticeable drops in accuracy and efficiency, indicating that the current decay structure contributes meaningfully to the model’s performance.\n\nWe acknowledge that a more targeted ablation of decay configurations (e.g., fixed vs. learned $\\alpha$) would provide further insight, and we will consider including this in future iterations.\n\nWe will also clarify the initialization and training behavior of $\\alpha_{ij}$ in Appendix C.4: these are initialized to small positive values and optimized via gradient descent, with implicit control via regular training dynamics.\n\n---\n\n>**Additional limitations: CPU latency, noise sensitivity, and continual learning**\n\n* Latency on CPUs and edge devices: You’re right—while FLAME achieves real-time throughput on modern GPUs, it does not yet meet real-time constraints on CPU-only systems (as shown in Table 8). FLAME’s current design targets embedded accelerators or GPU inference settings; further optimization (e.g., model pruning, integer quantization) is needed for ultra-constrained edge deployment. \n\n* Sensitivity to noise in event-by-event processing: This is a known trade-off of event-driven models. We mitigate noise sensitivity via multi-timescale filtering in the Event Attention Layer, but agree that high-SNR environments still pose a challenge.\n\n* Lack of continual learning: Correct—our current framework is designed for static offline training. We view on-device continual learning as an important next step, especially for long-term deployment scenarios."}, {"review_id": "457IYQ1Ukv", "reviewer": "Reviewer_r7Az", "summary": "1. The paper focuses on a basis and important topic for event-bases vision. The motivation is good.\n2. The proposed FLAME is technically sound. The methods seems effectively combines the event modality and the SSM structure. From the results, the FLAME shows its powerful  capability on balancing the compuatational efficiency and the task performance. There are plenty of theoretical analysis for proving the feasibility of the method.\n3. The paper presents good writing and formatting.\n4. The experiments are thorough and effective.", "questions": "As listed in the \"Strengths and Weaknesses\".", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The paper focuses on a basis and important topic for event-bases vision. The motivation is good.\n2. The proposed FLAME is technically sound. The methods seems effectively combines the event modality and the SSM structure. From the results, the FLAME shows its powerful  capability on balancing the compuatational efficiency and the task performance. There are plenty of theoretical analysis for proving the feasibility of the method.\n3. The paper presents good writing and formatting.\n4. The experiments are thorough and effective. \n\nWeakness:\n1. The length of each section and the formatting of some experimental results can be refined. For example, the section 4 would benefit from tighter editing, and some important results of ablation studies should be moved to the main manuscripts.\n2. Using LIF neurons as processing unit in the event attention layers raises my doubts. the leaky neurons is more difficult for model training compared with IF neurons. Besides, considering that the event data are sparse, especially under in  small motion environments or dark environments, the LIF neuron may leads to stronger sparsity. \n3. It seems that the paper lacks results of CIFAR-100 dataset. If the results are presented, please highlight the results. The FLAME presents its ability in some common datasets. I am wondering if the FLAME can be scaled to more difficult tasks like object detection/trakcing, motion estimation, depth estimation.", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for the encouraging evaluation and thoughtful feedback. Below we address the specific concerns raised:\n\n---\n\n> **1. Section length and formatting of experimental results**\n\nWe agree that Section 4 (Experiments) can be streamlined for clarity. Due to space constraints, we deferred several important ablation studies (e.g., NPLR impact, EAL removal, spatial pooling variations) to the appendix. If the paper is accepted, we will restructure Section 4 by condensing textual descriptions and **moving key ablation results (e.g., Table 7 from Appendix B.4)** into the main manuscript. We also plan to improve figure formatting (axis markers, font size) to enhance readability.\n\n---\n\n\n> **2. Use of LIF neurons and sparsity concern in low-activity regimes**\n\nWe appreciate the reviewer’s concern regarding training difficulty with LIF neurons. While it is true that LIF dynamics introduce decay and potential sparsity, we specifically use a **multi-branch EAL** design with **learnable time constants** $\\tau_d$ to counteract this. Each branch filters input events at different temporal scales, and the soma integrates across branches, ensuring robustness even under sparse or low-motion conditions. Empirically, FLAME performs well across both high- and low-activity datasets (e.g., SHD vs. CeleX-HAR), suggesting that the learned dynamics handle a wide range of temporal densities. We found that simpler IF neurons led to unstable training in deeper setups due to lack of temporal gating.\n\n---\n\n\n> **3. CIFAR-100 and scaling to harder tasks**\n\nThank you for raising this. We clarify that **CIFAR10-DVS** and **N-Caltech101** are standard benchmarks for event-based classification and are included in our evaluation (Table 2). To the best of our knowledge, **CIFAR-100 does not currently exist in event-based format** (e.g., from DVS or simulator pipelines), which is why it is not included. Regarding scaling FLAME to more complex tasks such as detection or depth estimation: while this paper focuses on classification and gesture recognition, the **asynchronous EA-HiPPO core and sparse, long-range memory design** are well-suited for extension to structured prediction tasks. We consider this a promising direction for future work.\n\n---\n\nWe are grateful for the reviewer’s comments and are confident that the final revisions will address these concerns. Thank you for considering an upgrade in recommendation."}, {"review_id": "H97a0UkCZr", "reviewer": "Reviewer_D3mP", "summary": "The paper introduces a SSM-based framework for event-based processing applied to vision tasks. It introduces 3 main contributions:\n\n-\tAn event attention layer, made with multi-timescale LIF neurons ;\n-\tAn event-aware HIPPO, with a memory retention moduled with inter-events intervals ;\n-\tA further optimization of the HIPPO with Normal-Plus-Low Rank (NPLR) decomposition.\n\nThe paper exhibits impressive results over the state-of-the-art notably by looking at the accurary vs computation cost ratio.", "questions": "My main question is about the provided FLAMES_model.py in the supplementary material: this implementation does not reflect at all everything described in the paper, yet seems to be provided as a way to reproduce the experiments. Is it on purpose? Do you plan to release a complete open source version of your framework? As is, this script only cast doubts about the implementation of your method, unfortunately.\n\nDespite the extensive supplementary material, I believe some critical information on how to choose or learn hyperparameters is missing, making results reproductability potentially very difficult. I would have appreciated a section dedicated to the training of the model.\n\nAlso, I would very much appreciate if the issues mentioned in the previous section would be answered as well.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "While the paper is well and extensively positionned against the state-of-the-art, both in methodology and in results, I found several issues about the methodology presented and its implementation:\n\n-\tThe connectivity of the event attention layer is not clearly explained, nor how its learnable timing factor are learned. It is therefore not clear as well how the spatial pooling works, as it is defined relative to the input but applied after the event attention layer. Furthermore, I could not see where spatial pooling is actually implemented in the FLAMES_model.py available in the supplementary materiel package.\n-\tIt is not clear what brights the coupling strength parameters (of branch to soma), as it seems it could be fused in the weights of the branches current ;\n-\tNot much detail is given about the NPLR decomposition used: it is not explained how it is obtained and with what parameter (notably rank). Furthermore, I failed to see any implementation of this mechanism in the provided FLAMES_model.py. The NPLR decomposition is presented as being fundamental to the computation efficiency of the method, yet no impact analysis or ablation study is provided (a single number is given in the supplementary material l.208, without any detail on the setup of the NPLR).\n-\tIt is not specified how the HIPPO A matrix is initialized and learned such that the condition on its eigen values (<0) remains true.\n\nHowever, a lot of extra information is provided through the supplementary materiel, which includes detailed approximation and numerical stability proofs, as well as more details on the implemented method. This is definitely a strong point.", "was_revised": true, "final_justification": "I raised my rating to 4, considering that the authors gave sufficient transparency and openness garantees to ensure the reliability and reproductibility of their results, which are solid and with a potential of moderate-to-high impact on one sub-area of AI (event-based vision).", "rebuttal": "We sincerely thank the reviewer for the encouraging evaluation and thoughtful feedback. Below we address the specific concerns raised:\n\n---\n\n> **Clarification on Event Attention Layer Connectivity and Spatial Pooling**\n\nWe thank the reviewer for the detailed feedback and the opportunity to clarify these design components.\n\n1. **Connectivity of the Event Attention Layer (EAL)**:\n   The EAL comprises LIF neurons augmented with *multiple independent dendritic branches*, each associated with a distinct time constant \\$\\tau\\_d\\$. Each branch integrates inputs with its own temporal decay (see Equation (5), Appendix C.2, and Algorithm 3, p.24–25):\n\n   $$\n   i_d(t + 1) = \\alpha_d i_d(t) + \\sum_{j \\in \\mathcal{N}_d} w_j p_j \\quad \\text{with } \\alpha_d = e^{-1/\\tau_d}\n   $$\n\n   These branch currents are aggregated at the soma via learnable coupling strengths \\$g\\_d\\$:\n\n   $$\n   V(t+1) = \\beta V(t) + \\sum_d g_d i_d(t)\n   $$\n\n   This architecture enables the neuron to simultaneously process short- and long-range temporal patterns while maintaining event-level sparsity. The learnable \\$\\tau\\_d\\$ are updated via backpropagation (standard autograd), and the \\$g\\_d\\$ provide independent control over temporal feature relevance.\n\n2. **Role and Timing of Spatial Pooling**:\n   The spatial pooling layer is applied *after* the EAL, operating over the sparse event map \\$I(x, y, t)\\$ generated by LIF outputs (see Algorithm 4, Appendix C.3, p.25). Pooling is performed per event timestamp using a local max operation (e.g., 2×2 window), as shown in:\n\n   $$\n   I_{\\text{pooled}}(x', y', t) = \\max_{(x, y) \\in P(x', y')} I(x, y, t)\n   $$\n\n   This preserves temporal resolution while reducing spatial redundancy.\n\n3. **Clarification on Code (FLAMES\\_model.py)**:\n   The provided script is a simplified abstraction to illustrate architectural flow. It excludes some modules (e.g., pooling logic and low-level event branching operations) for brevity. The full implementation, including EAL with branch-level timing and spatial pooling, will be released in full alongside the final paper for full reproducibility.\n\nWe will explicitly clarify this in Section 3 and Appendix C.2–C.3 in the camera-ready version to avoid further ambiguity. Thank you again for the helpful critique.\n\n---\n\n>**Role of Coupling Strength Parameters Between Branch and Soma**\n\nIn our model, the coupling strengths $g_d$ between dendritic branches and the soma are explicitly separated from the synaptic weights $w_j$ for both functional and practical reasons.\n\n1. Functional Separation:    Synaptic weights $w_j$ determine how input events influence each branch’s current $i_d(t)$, while the coupling parameters $g_d$ control the *relative contribution* of each branch to the soma membrane potential $V(t)$. This is described in **Equation (5)** in **Appendix C.2**, and enables the model to:\n\n   * Learn branch-specific temporal filtering via $\\tau_d$ and $w_j$,\n   * Independently learn how much each timescale branch should contribute to final firing via $g_d$.\n\n   Fusing these into a single weight would reduce this expressive flexibility, especially in multi-timescale scenarios.\n\n2. Empirical Justification:    In early experiments, we explored absorbing $g_d$ into $w_j$, effectively collapsing the dendritic integration into a flattened LIF structure. This led to reduced stability and slight performance degradation, particularly in models with 3 or more branches. Keeping $g_d$ separate improves training stability, gradient flow, and enables selective modulation of timescales at the soma level.\n\nWe will clarify this rationale more explicitly in Section 3.1 and reinforce it in Appendix C.2. Thank you again for prompting us to expand on this design choice.\n\n---\n\n> **Details and Implementation of NPLR Decomposition**\n\n We’re sorry if the description of the NPLR mechanism was not sufficiently clear in the main text. Key details are included in **Appendix C.4 (pp. 26–29)**, but we agree this deserves more visibility and clarification in the main paper also.\n\n1. **Purpose and structure**\n   We use the **Normal Plus Low-Rank (NPLR)** decomposition to factor the state matrix $A$ in the EA-HiPPO core as:\n\n$$\nA = V \\Lambda V^* - P Q^*\n$$\n\nHere, $\\Lambda$ is a diagonal matrix with **negative real eigenvalues** to ensure stability, $V$ is a fixed DFT basis, and $P, Q \\in \\mathbb{C}^{N \\times r}$ are learnable low-rank correction matrices. This formulation enables **efficient convolution via FFT** and significantly reduces compute and memory from $O(N^2)$ to $O(Nr)$.\n\n2. **How it is initialized and trained**\n   We use $N = 64$ and fix the rank to $r = 8$ in all experiments. As noted in Appendix C.4, $\\Lambda$ is initialized with small negative values (e.g., sampled from $[-1.0, -0.1]$) and updated during training with gradient descent. This ensures the eigenvalues of $A$ remain strictly in the left half-plane. The $P$ and $Q$ matrices are also learned and jointly optimized with the rest of the model.\n\n3. **Efficiency impact**\n   The role of NPLR is evaluated in **Appendix B.4, Table 7**. Disabling NPLR increases FLOPs by 2.1× and GPU memory usage by \\~38%, without improving accuracy. This confirms that NPLR is essential to the model’s **efficiency and scalability**, especially in long-sequence, real-time settings.\n\n4. **Code clarity**\n   You’re absolutely right that `FLAMES_model.py` does not reflect this logic—it’s a simplified script provided to show the architectural flow. We apologize if that caused confusion. The **full implementation of the NPLR logic**, including FFT-based kernels, will be released with the final paper, along with complete training and evaluation scripts.\n\nWe will revise Section 3.3 and highlight the relevant content in Appendix C.4 more clearly in the final version. Thank you again for pushing us to improve the clarity of this key module.\n\n---\n\n> **Initialization and Stability of the HiPPO Matrix $A$**\n\nThank you for this thoughtful comment. We apologize if our description of the initialization and stability conditions for the HiPPO matrix $A$ was not sufficiently clear in the main text.\n\nTo clarify: the matrix $A$ used in our EA-HiPPO core is initialized using the **HiPPO-LegS formulation**, which guarantees that all eigenvalues satisfy $\\text{Re}(\\lambda_i) < 0$. This ensures stability of the continuous-time system. We mention this in **Appendix A.2 (p.11)**:\n\n*“Given that $A$ is a HiPPO matrix, all its eigenvalues satisfy $\\text{Re}(\\lambda_i) < 0$… This implies that $A$ is a Hurwitz matrix, ensuring that the system is asymptotically stable.”*\n\nThis property is preserved when moving to our **event-adaptive variant** $A_S(\\Delta t) = A \\circ F(\\Delta t)$, since the decay modulation matrix $F \\in (0,1]$ does not flip the sign of eigenvalues (Theorem 1, p.17).\n\nIn the **NPLR-based version** of EA-HiPPO (Appendix C.4), the eigenvalue matrix $\\Lambda$ is initialized with small negative values (e.g., $[-1.0, -0.1]$) to ensure the same property. When learned, we monitor or clip these values to keep the system contractive.\n\nWe appreciate the reviewer for highlighting this subtle but important point. If the paper is accepted, we will revise Section 3.3 to clearly surface this explanation and avoid further ambiguity.\n\n---\n\n> **Scope of `FLAMES_model.py` and Reproducibility**\n\nWe appreciate the reviewer raising this important point, and we apologize for any confusion the current `FLAMES_model.py` script may have caused.\n\nTo clarify: the script included in the supplementary material is **not intended to be a full implementation**, but rather a **simplified abstraction of the model’s architectural flow**. It was meant to convey the high-level composition (EAL → spatial pooling → EA-HiPPO), and intentionally omits several key components such as:\n\n* NPLR-based state-space kernels,\n* Full multi-branch LIF dynamics,\n* Spatial pooling internals,\n* Training infrastructure.\n\nWe completely understand the concern, and we agree that without this context, the script may give the impression of being incomplete.\n\nWe confirm that **the full codebase used to produce all experiments will be released** with the final version of the paper. This includes:\n\n* The complete EA-HiPPO module with NPLR and FFT support,\n* Full multi-branch EAL implementation with learnable decay,\n* Spatial pooling and streaming inference code,\n* Scripts for training, evaluation, and reproduction across all benchmarks.\n\nWe will clarify this intent explicitly in the README and in the main text to avoid further confusion. We thank the reviewer again for bringing this up—it helps us strengthen the transparency and usability of our work.\n\n---\n\n> **Training Procedure and Hyperparameter Selection**\n\n We agree that providing clear guidance on training and hyperparameter selection is essential for reproducibility, and we apologize if the current presentation made this difficult to extract.\n\nWhile training details are provided in **Appendix C.1**, including optimizer choice, learning rates, batch sizes, and sequence lengths for each dataset, we recognize that this information is somewhat fragmented and lacks the dedicated structure the reviewer rightly expected.\n\nTo clarify:\n\n* All models were trained using the Adam optimizer with a learning rate of 1e-3 (decayed to 1e-4), batch sizes of 16–32 depending on dataset size, and sequence lengths of 200–600 timesteps.\n* Architectural hyperparameters such as the number of EAL branches (typically 3), decay initialization ranges ($\\tau_d \\in [1, 50]$), NPLR rank ($r = 8$), and state dimension ($N = 64$) were selected via coarse grid search and kept fixed across datasets once validated.\n* Gradient clipping (1.0) and standard weight initialization were used to ensure stability.\n\nWe will consolidate these training choices into a dedicated section in the appendix (or the main paper, if space allows) and accompany our code release with full config files to enable straightforward reproduction."}]

Author rebuttal:

Meta-review: This paper received four mixed, but largely positive-leaning reviews initially. 

There was general appreciation for the importance of the problem considered here, the technical soundness of the approach, and the performance gains (in terms of computational complexity) demonstrated with the proposed method.

There were some concerns raised in the initial reviews regarding experimental evaluation and presentation quality. There was significant author-reviewer discussions following the initial review period, after which the initially negative reviewer raised their rating to a borderline accept, and therefore all four reviewers converged to a positive (Borderline Accept) rating – and an accept decision was reached.
Decision: Accept (poster)