SAFA-SNN: Sparsity-Aware On-Device Few-Shot Class-Incremental Learning with Fast-Adaptive Structure of Spiking Neural Network
Authors: Huijing Zhang
Muyang Cao
Linshan Jiang
Xin Du
Di Yu
Changze Lv
Shuiguang Deng
Venue: ICLR 2026 Poster (2026)
Keywords: Few-Shot Class-Incremental Learning
Spiking Neural Network
Brain-Inspired Learning
Edge Computing
Primary area: applications to neuroscience & cognitive science
TL;DR: An SNN-based approach for on-device few-shot class-incremental Learning with practical implementations on edge devices.

Abstract: Continuous learning of novel classes is crucial for edge devices to preserve data privacy and maintain reliable performance in dynamic environments. However, the scenario becomes particularly challenging when data samples are insufficient, requiring on-device few-shot class-incremental learning (FSCIL). Although existing work has explored parameter-efficient FSCIL frameworks based on artificial neural networks (ANNs), their deployment is still fundamentally constrained by limited device resources. Spiking neural networks (SNNs) process spatiotemporal information efficiently, offering lower energy consumption, greater biological plausibility, and compatibility with neuromorphic hardware than ANNs. In this work, we propose an SNN-based method containing Sparsity-Aware neuronal dynamics and Fast Adaptive structure (SAFA-SNN) for on-device FSCIL. By threshold regulation, most neurons exhibit stable spikes and others exhibit adaptive spikes. As a result, synaptic traces that encode base-class knowledge are naturally preserved, thereby alleviating catastrophic forgetting. To cope with spike non-differentiability in backpropagation, we employ a gradient-free technique, i.e., zeroth-order optimization. Moreover, class prototypes can limit overfitting on few-shot data but introduce bias. We enhance prototype discriminability by orthogonal subspace projection. Extensive experiments conducted on two standard benchmark datasets (CIFAR-100 and Mini-ImageNet) and three neuromorphic datasets (CIFAR10-DVS, DVS128 Gesture, and N-Caltech101) demonstrate that SAFA-SNN outperforms baselines, specifically achieving at least 4.01\% improvement at the last incremental session on Mini-ImageNet and 20\% lower energy cost on CIFAR-100 over baselines with practical implementation.

Published as a conference paper at ICLR 2026
SAFA-SNN:
SPARSITY-AWRR ON-DEVICE FEW- SHOT CLASS-INCREMENTAL LEARNING WITH FAST- ADAPTIVE STRUCTURE OF SPIKING NEURAL NET-
WORK
Huijing Zhang'
Muyang Cao Shuiguang Deng'
Linshan Jiang
Xin Du
Di Yu²
Changze Lv
Zhejiang University
Shanghai Innovation Institute
National University of Singapore
Fudan University
{huij muuijingzhang muyange, xindu, yudi2023, dengsg] eduu. cn Linshan@nus. edd.sseduusss sg
czlv240m. fudan edu. en
ABSTRACT
Continuous learning of novel classes is crucial for edge devices preserve data privacy and maintain reliable performance in dynamic environments. However, the scenario becomes particularly challenging when data samples are insufficient. requiring on-device few-shot caas-incremental learning (FSCIL).. Although exist- ing work has explored parameter-efficien FSCIL frameworks based on artificial neural networks (ANNs), their deployment is still fundamentally constrained by limited device resources. Spiking neural networks (SNNs) process spatiotempo- ral information efficiently, offering lower energy consumption, greater biological plausibility, and compatibility with neuromorphic hardware than ANNs. In this work, we propose SNN-based method containing Sparsity-Aware neuronal dy- namics and Fast Adaptive structure (SAFA-SNN) for on-device FSCIL By thresh- old regulation, most neurons exhibit stable spikes and others exhibit adaptive spikes. As a result, synaptic traces that encode base-class knowledge are natu- rally preserved, thereby alleviating catastrophic forgetting. To cope with spike non-differentiability in backpropagation, we employ gradient-free technique, i.e., zeroth-order optimization. Moreover, class prototypes can limit overfitting on few-shot data but introduce bias. We enhance prototype discriminability by orthogonal subspace projection. Extensive experiments conducted on two stan- dard benchmark datasets (CIFAR-100 and Mini- Mini-ImageNet) and three neuromor- phic datasets (CIFARIO-DVS, DVS128 Gesture, and N-Caltech101) demonstrate that SAFA-SNN outperforms baselines, specifically achieving at least 4.01 % im- provement at the last incremental session on Mini-ImageNet and 20% lower en- engy cost CIFAR-100 over baselines with practical implementation,
INTRODUCTION
Intelligent edge devices sense dynamic contexts in their surrounding environment and make deci- sions based on sensor data (Ravaglia et al., (02T). The high labeling cost and privacy constraints make it difficult to acquire sufficient data for continual batch learning (Wang et al. 20226). To this end, on-device Few-Shot Iaasemnncrmmtttl Learning (FSCIL) introduced to incrementally learn from newly collected sensory data with scarce labeled samples while preserving existing knowledge. Spiking Neural Networks (SNNs) (Maass] 1997) are attractive for edge devices (Deng et al. 2025).
neurons fire only upon event-driven activation, reducing unnecessary computation cost (Degg
et al. 2020 Lv et al. 2024).
"Corresponding authors.
Codes available at wtttppp://sthbb.cccm////guuuuuup SNN

---

Published as a conference paper at ICLR 2026
Edge Device
Abundant Base Data Training
Few-shot
Few-shot Class Incremental Learning
Span Years
Data
Model
Algorithm
SNN Neuron
Spana degle-it mustio
ANN Neuron
Donos Beatingpdee muttylication bellags
accumalation redoce
Xisk As
Muki- bak Values X
SI
5p
Bimarr Spile
Xit
Figure I: Upper. The on- Device FSCIL scenario includes three stages: Abundant base data training, Few-shot data collection, and Few-shot dlass-incremente learning. Bottom. Compared to the ANN neuron, the SNN neuron communicates by spikes, showing memory and computation benefits.
However, on-device FSCIL faces two significant challenges, i.c., catastrophic forgetting (French) 1999) and overfitting (Tao et al. 2020). Catastrophic forgetting reflects the model's 's difficulty learn- ing new tasks while retaining information of originally learned tasks, necessitating a delicate bal- between plasticity and stability. Overfitting occurs when the model memorizes limited training samples, leading to poor generalization. On the one hand, recent advanced solutions tackle these challenges by parameter-efficient fine-tuning (PEFT) (Liu al. 2024a), which fine-tunes parame- ters on top of a large frozen pre-traing pre-trained model. Nonetheless, these approaches remain ineffective under limited memory budgets (i.e., 4-12 GB) (Ma al., 2023). On the other hand, concurrent on- device SNNs concentrate on integrating SNN algorithms with specialized neuromorphic hardware, which shifts the consideration of the above problems the later model. Additionally, most existing neuromorphic systems are still trained offline and remain static during development (Safa 2024 presenting heavy dependency on abundant labeled data.
Besides, smart devices are severely constrained in terms of memory capacity and maximum perfor- mance (Shuvo et al., 2022), extremely scarce labeled samples, and the persistent need for model up- dates. Edge devices incapable of accumulating comprehensive data on all classes due to limited storage capacity and must simultaneously perform low-power inference while continuously learning from real-time data streams. We provide the process of on-device FSCIL scenario and compare the behavior of SNN and ANN neuron in this scenario in Figure] |
To tackle the above challenges, propose Sparsity-Aware neuronal dynamics and Fdat-Ada prototype subspace projection with SNN (SAFA-SNN) for general on-device FSCIL. Sparsity mea- sures the spike frequency of neurons. Neurons with dynamic thresholds defined stable or active, where most remain stable and only a few are active for new tasks, minimizing updates to the majority of synapses. Zeroth-order optimization a gradient-free method approximate true gradi- ents using only function values (Liu et al. (2020b), which enables SNNs to handle non-differentiable spike activation while avoiding limited width of surrogate gradients (Lian et al.. (023). In incremen- tal learning sessions, retraining the model with few-shot samples is inherently prone to severe over- fitting. To migrate this issue, we project the prototypes onto the subspace spanned by base classes, enhancing the discriminability of new classes. We conduct practical implementation and extensive experiments on static and neuromorphic datasets, demonstrating the performance of SAFA-SNN.
The main contributions of this paper are follows:
We emphasize practical challenges of on-device FSCIL, where both training and inference are performed essurcce-constraiig devices using energy-efficiest mendware-friendee SNNs, contrasting with ANN-based FSCIL that depends on large-scale server models.
We propose a sparsity apare aware FSCIL method, called SAFA-SNN, with three key features. First, incorporating spikes for prediction, we propose the Sparsity-Aware dynamics for SNN. Then, to solve the problem of non-differentiable spikes in back propagation, we adopt zeroth-order optimization within neuronal dynamics. Finally, we enhance the dis-
2

---

Published as a conference paper at ICLR 2026
criminability of prototypes by subspace projection in incremental learning. To the best of our knowledge, this is the first SNN-based solution towards general on-device FSCIL.
To prove the effectiveness of our proposed SAFA-SNN for on-device FSCIL, we imple- ment SAFA-SNN on realistic devices (i.e., Jetson Orin AGX), and evaluate on five datasets (CIFAR-I00 MinilmageNet, CIFARIO-DVS, DVS128 Gesture, and N-Caltech101) and various models (Spiking VGG, Spiking ResNet, and Spikingformer), demonstrating that SAFA-SNN achieves excellent performance and notable sparsity advantages. We further evaluate the actual on-device energy consumption, demonstrating our efficiency advantage.
RELATED WORK
2.1 FEW-SHOT CLASS-INCREMENTA, LEARNING
To the best of our knowledge, existing methods for few-shot itassmnncrementll learning (FSCIL) are mostly ANN-based, with little exploration in SNNs. The primary work (Tao et al.) 2020) preserves feature topology using a neural gas network. Researchers (Zhang et al., [021] model prototypes
as nodes in graph attention network to propagate context information. Existing methods (Zhou
jet al., 2022] Wang et al. 2023) commonly adopt prototypes to replace traditional MLPs and refine them (Song et al. 2023 Some studies (Shi et al., 2021 Liu al. (2023) tackle class imbalance by exploring optimal base class distributions. Recent advances (Park et al., 2024, Sun et al.) [024] Wang et al. 2024) train few parameters (prompts) from the frozen pre-trained model, achieving devices. Prior work focuses on a specific processor (Huo et al. 2025). while we explore FSCIL on general devices.
high accuracy. However, memory overhead limits their use for resource-constraiig
2.2 SPIKING NEURAL NETWORKS WITH DYNAMIC THRESHOLD
In the realm of SNNs, the dynamic threshold mechanism aims to manipulate the threshold spon- taneously. BDETT computes thresholds via average membrane potentials for neuronal homeosta- sis (Ding et al. 2022), but excessive spiking remains. Highly active neurons skew the average, causing aggressive firing and reduced sparsity. LTMD learns initial membrane potentials for heuris- tic neuron thresholding (Wang et al. 2022a), but neuron behaviors across layers remain interde- pendent in spatial and temporal processing. Soft threshold schemes enable adaptive layer- wise sparsity (Chen et al. (003)). but independent pruning causes error accumulation, affecting overall model performance. Efforts with combinations of adaptive threshold methods have led to innovative approaches (Wei al. 2023 Hao et al., 2024). Our paper further tackles FSCIL challenges via sparsity- aware dynamic thresholds that shift networks to task-specific local contexts.
2.3 ON-DEVICE SNN TRAINING AND INFERENCE.
Previous studies on-device SNNs can be classified into two categories: inference and training with updating. The lightweight SNN is a crucial application for resource-constraing edge de- vices Yuu et al. 2024ap. For instance, Lite-SNN (Liu al.] 2024b) proposes spatial-temporal compression strategy to reduce memory and computational cost. Most researchers focus on effi- cient quantization for SNN-training like SNN pruning (Wei al.] 2025 Qiu al. 2025 Yu al.) 2025), to facilitate efficient deployment. Current SNN training and updating focus on gradient up- date (Anumasa al., 2024) local SNN training optimization (Mukhoty
al. 2023 Yu et al. 2024b) to enable low-latency SNN training on devices. In this work, we explore both SNN-batch training on abundant base classes and inference in FSCIL phases, which typically occur in the over- changing on-device environments, achieving high performance and low energy consumption.
PRELIMINARIES
3.1
SPIKING NEURAL NETWORK
We use the leaky integrate-and -iie (LIF) neuron model, a simplified but common approach to neu- romorphic computing that mimics biological neurons through charging, leakage, and firing. The
3

---

Published as a conference paper at ICLR 2026
process of state updating in LIF neurons follows certain principles:
U/ = + 4
(1)
S: = 11, Uth)
(2)
U, - S.U + (1-S.)U;
where H LU U Y the time constant, I the spatial input calculated by applying function f with x as input and f as learnable parameters, and U is temporary voltage at time step t, whose value instantly changes. At time step I, when the membrane U/ rises and reaches the threshold Uth. a spike denoted by S, is generated as 1, and U; will be reset to U ; otherwise, S, remains O. Afterwards, U² changes to U, SNNs present challenges in terms of the non- dinnffferentiable activation function in Equation 2 The backpropagation of directly training SNNs can be described as
δL aw
dL as, OU, OL as, au, a., aw'
is non-differentiable, which is typically replaced by sur- rogate gradient (SG) methods (Wu et al. 2018). However, SG deviates significantly from the true gradients and easily causes vanishing gradient problem (Neftei et al. (019). To address this issue, we employ zeroth-order optimization to estimate the gradients, inspired by Mukhoty et al. (2023)
where W is the weight of input, and 0S. D0,
3.2 CLASS PROTOTYPE
Class prototypes are widely used in few-shot learning based on the observation by Snell al. (2017). In FSCIL, researchers use prototypes as dynamic classifiers (Zhang et al.] [021]. In detail, they decouple the FSCIL model into a feature encoder (o with parameters and linear classifier W. and freeze (o(+) during incremental sessions, with only the classifier being updated. For each new class i, the weight W; in the classifier is parameterized by the mean embeddings of each class,
i... the prototype Pt, which can be denoted as P. = K EjPrl ((,)=))((z)), where K denotes the number of samples in the class i.
4 ON-DEVICE FSCIL PROBLEM
On-device FSCIL problem depicts data with scarce samples always coming, models need maintain performance on all seen classes while maintaining low latency and high security, enhanced data privacy, and real-time decisions under resource- -imited edge devices. The data resources are always scarce, for example, users take an average of 4.9 photos per day (Gong et al. 2024).
Assuming FSCII, tasks contain base session and a sequence of S incremental sessions, the train- ing data available for each session can be denoted as (Dirain, Dfrain ...., Digain). Dirain
IDggruin where x² E X, is the training instance and vf E Y, is its corresponding label in session s. The classes in each session s is disjoint with another session s', i.e., Dérrin n Dérain = 0. The training data in session 0 have abundant instances while each incremental session has much smaller instances, i.c., | >> ! : Each incremental session has training sets in the form of N-way K-shot, i.e there are N classes and each class has samples. In session 8, the model's 's per- formance is assessed on validation sets from all encountered datasets (S 8) to minimize the expected risk R(f s) over all seen classes Zhang et al. 2025):
{(x], vf)) 1
1==
E E (
[ (f(x;;; Drrain (0-1),
(5)
where the algorithm f constructs new model using current training data Dirain and parameters g--1 to minimize the loss L. The general goal is to continually minimize the risk of each new session, i.c., R(f,)) Note that in this paper, "session" and "task" are used interchangeably.
In many real-world scenarios, when FSCIL applications are deployed on resource- edge de- vices, the learning efficiency w.r.t. both few-shot inference speed and memory footprint becomes crucial metric. Being able to quickly adapt deep prediction models in low computational cost at the edge is necessary to better suit the needs of personal mobile users. However, these aspects are rarely explored in prior FSCIL research, which motivates us to solve these on-device FSCIL challenges beyond catastrophic forgetting and overfitting.
4

---

Published as a conference paper at ICLR 2026
logits
U
Sx
Hlse
1000
+
logits
Nmw Class
W
mask
New
18
(a) Sparsity-Aware Neuron Dynamics
(b) Zeroth-Order Optimization
(c) Prototype Subspace Projection
Stable Neurons
Active Neurons
Sampled Points
Subspace Projection
Orthogonalization
Figure 2: SAFA-SNN framework includes three main components: (a) Training abundant data and selecting active and stable neurons by masks. (b) Top: forward propagation through FSCIL process; Bottom: backpropagation using zeroth-order optimization only the base class training. Freez- ing backbones and updating the prototypes by subspace projection in the incremental inference.
METHODOLOGY
5.1 MOTIVATION
While considerable research focuses on soft-subnetworks that freeze the major part of the model parameters to preserve learned knowledge and train minor part of parameters for the selected subnetwork in incremental learning sessions (Mazumder et al., [2021] Kang et al.) [2023), inspired by the regularized lottery ticket hypothesis, the question of expanding the network structure remains unsolved. Additionally, biological neural networks are interconnected through synapses, receiving input spikes and emitting output spikes according to the membrane accumulation, forming different subnetworks by synapse weights, but different regularized subnetworks formed by different spiking firing dynamics in neurons remain unexplored. Motivated by these challenges, we explore the help the trade-off between plasticity and stability in dealing
regulation of neuronal dynamics with FSCIL tasks. Despite ongoing efforts, overfitting remains a challenge. Subspace projection, using class-specific matrices, offers robustness few-shot learning (Simon al. 2020). Despite previous studies explore prototype rectification methods by trainable networks (Tang et al. 2024) and training-free free calibration (Wang et al., 2023), the prototypes of novel classes still tend to deviate to the base one. These motivate the need for the subspace projection scheme that represents each new class with an orthogonal subspace basis and projects onto the most similar base classes.
5.2 OVERVIEW OF SAFA-SNN
SAFA-SNN for on-device FSCIL comprises three parts: sparsity- dynamic mechanism to mitigate forgetting, the formulation of zeroth-order optimization for di-diffeereniaale spikes, and a prototype subspace projection for fast adaptation on few-shot new class. Figure overview of SAFA-SNN in FSCIL, and the full FSCIL procedure is detailed in Algorithm
2 illustrates the
5.3 SPARSITY-AWARE NEURONAL DYNAMICS
Previous studies have demonstrated the functional role of neuronal firing activity visual classifi- cation learning (Freedman et al.) 2002). The sparsity-aware neuronal dynamics implement efficient heterogeneous plasticity in SNNs by a dynamic threshold function. involves neuron division and dynamic threshold mechanism built the LIF neuron model. Following homeostatic plasticity rule (Turrigiano) 2012). stable neurons maintain firing rates that stay close to the firing rate during base-class training. which helps preserve previously acquired knowledge. Neurons in each channel are randomly designated as either stable or adaptive, using a channel-wise mask M, which is de- fined as M {m,)ee c 1,,,... C, where is the channel number and m. adaptive if m 1. otherwise they
[y0), where
n (0, is the adaptive ratio, and neurons in channel are stable. Most neurons remain stable to maintain consistent spike patterns, while others adapt to incremental learning. We define the sparsity channel-wise firing rate with spike output as
5

---

Published as a conference paper at ICLR 2026
Algorithm Overall SAFA-SNN Algorithm
Input: Dataset {D, J membrane U, feature extractor ø Output: Spike trains St, parameters θ
1: Initialize mask M to set 17% adaptive neurons and setting others 0 in each channel
2: Initialize threshold Uth with zeros
3: for session
to do
4: Conduct state updating in LIF neuron models by Equation and
5: if == 0 then
6: Calculate the threshold regulation factor A by Equation Sample point ~ (0, 1), 1,,, 1,,...,b
7:
8:
|x| < 6|2z| then
9:
Estimate the gradient 9:
|z4
end if
11:
Estimate the non-differential as: by Equation 10 au. Conduct base session optimization L and update (0)) by Loss in Equation 13
12:
13:
else
14:
Update threshold Uth by U Uth A(r. r%)
15:
Compute the projection subspace by Equation (14)
16:
Conduct incremental learning with P Equation 15
end if
18:
end for
[] Ea.t.neen S(b,t,n). Define the threshold regulation factor
A = 31 - M) + yM,
(6)
where / and y are ppper-parameter, and setting 3 > 7 ensures that thresholds adaptive neurons change more freely than those of stable neurons. The threshold is dynamically updated by
U th Uth + A(r.- -rb),
where r. and r4 are the sparsity of neurons current incremental task and base task 0, respectively. With dynamic thresholds, neurons maintain relatively stable spike patterns, which limits unnecessary synaptic updates. As a result, synaptic traces encoding base-class knowledge are preserved. The whole neuronal dynamics process is provided in Appendix A
5.4 ZEROTH-ORDER OPTIMIZATION
Multi- Multi-point estimate. Zeroth-order optimization is beneficial to computation-difit difffccul or infeasible gradient problems, since it can approximate the full gradients or stochastic gradients through func- tion value (Liu et al., 2020b).. We start by presenting the gradient estimate of f f(r) with two-point directional derivative as
ff(x) = ((d) yb 1f(x +zz)) - f(x -zz,),
((d)))
(8)
where b is the number of i.idd. samples { and is a dimension-deppendent factor. Let gu) 88, with membrane potential = 2e denotes the central finite difference approximation JU, Then, for u E IR"
(u: H(u dz) H 82)
0. |u|
5 |z|
26
25 [u|
< 5
where z obeys a specific distribution p over empirical samples { and H is the heaviside function defined in Section Assessing whether perturbations 2ō and u - 20 induce spike in the neuron allows for an approximation of the gradient. Through random permutation, (u) accommodates the influence of all neighbors, for balanced depiction of local dynamics. The approximation (x at given input can be computed as
os, = aU, b Σg
1 b
g² (u; 5, 2.).
(10)

---

Published as a conference paper at ICLR 2026
Gradients are typically approximated by averaging multiple perturbed samples, where each pertur- bation produces local gradient, resulting in a more reliable estimate (See Appendix B).
Convergence analysis of ZOO. We start with some conventional criteria for convergence analysis in zeroth-order optimization literature (See Appendix C.. The von-differentiabiliyy of Equation 2
renders the entire optimization problem non-convex. The convergence is evaluated using the first- order stationary condition via the squared gradient norm (Proof in Appendix D
E [1b[))) 0(8² + 3
(11)
where denotes the update iteration. The convergence upper bound
Blg(ur) -(()] 0
(12)
Moreover, more details on analysis of sources of error in ZOO are provided in Appendix
Loss function. The total loss function, incorporating both the temporal error term (TET) Deng et al. 2022) and the MSE loss, given by
(1 Σ 1=1
CCE(ue, y) + XLMSE(u,, y),
(13)
where T denotes the number of time steps, and À is hyperparameter that governs the relative contribution of the temporal prediction error term and LMSE.
5.5 FAST ADAPTIVE PROTOTYPE SUBSPACE PROJECTION
Class prototypes averaging extracted features often cause discrepancies with actual data distribu- tions. Prototypes learned from the scarce and
samples of novel classes are prone to bias in FSCIL (Liu et al. (2020a). To mitigate this bias, we introduce prototype orthogonal sub- space projection, which updates novel-class prototypes in two steps: (i) projecting novel-class pro- totypes onto the orientation subspace spanned by base-class prototypes; and (ii) blending the pro- jected prototype with the original novel prototype through convex combination. As denoted in Sec- tion 3.2 we further define two specific prototypes that belong C3 base classes and CN new classes
as B |P P
P
E ER
Bx and Č = | 2
P
P
RNND
P
respectively, where is the feature dimension, B [Cal, and N the subspace structure spanned by the base classes, construct to represent a generalized inverse of a covariance-like matrix in projection calculation, given by
CN To obtain information on
prototype representation G
G
'B.
(14)
Note that the normalization term B B is necessary because the generation process of the subspace basis is not guaranteed to be orthogonal to each other. Hence, we construct [Cal distinct orthogonal subspaces represented by G for base classes. As classes increase, diverse class information allows extending classifiers with prototypes. The projection vectors P proj are updated by mapping the coordinates in the base subspace back to the original D-dimensional space, denoted as P proj = ČG. Finally, we integrate base and new knowledge by reconstructing the new information within the base subspace by
(1 ---a)C+ aP PPPj..
proj.
(15)
where the trade-off coefficient a further balances the integration base and novel knowledge. The current classifier weights are updated by P to enable new class prediction (See Appendix for more analysis). The orthogonal projection measures the alignment between novel and base classes, where a larger projection magnitude implies stronger cosine similarity. Consequently, features with higher similarity contribute more to the prototype update. Compared to the raw prototype C, the updated prototype lies closer the expected semantic direction and captures more discriminative features.

---

Published as a conference paper at ICLR 2026
Table I: Comparison with SOTA methods on MinilmageNet dataset for On-Device FSCIL.
Accuracy eoch sesion (18)
Method
Aura
(SNN)
40.51gmm
46.52
44.28g
g6 uo
g5.78
40.18g
+12.52
FACT
101
16.30
12.30
40, 40.16...r
18/47
16.0
+11.00
16.32
31.17
20.00
27.30at.m
26.36.1
REDIST (SNN)
43.72gaum
40.00
57
35.51
33 at
30. Peaus
28 20
27 88
+20.85
SAYC (SNN)
41.7010.a
38 38.541000
35.79
10.40
11.3
29.47.300
27.5 17.80
26.37.1.00
25.00
32.17
20,66
TEEN (SNN)
60.87a1m
52 52.70g
16 16.0
48.8ta
42
30 30.18ar
38 ae
46 Milllgmm
+0.000
WARP (SNN)
50.07
51.67
29.58
28.07
26.28
22.44
22.18
21 31,30
27.47a
+27.00
CLOSER (SNN)
60.881cm
57.721em
54.741c.00
32.341o0r
40.881821
47.0016.16
46.941c.4
44.89
50.381000
14.00
ALADE (PSCIL)
17.91
47 1940
1000
41 11.75
30.771..00
37 1747
36.0
38.57
41 41.42
+11.0
SAFA-SNN
Luas 64.02...5
was
55.39
52.0010.0
39.4510u7
EXPERIMENTS
6.1 EXPERIMENTS SETUP
Datasets and Spiking architecture. We eval- uate the generalization performance of SAFA- SNN standard benchmark datasets, i.e., CIFAR- 100 (Krizhevsky et al., (2009) and Mini- ImageNet (Russakovsky al. 2015), each split into eight 5-way 5-shot incremental tasks. We also extend experiments on three neuromor phic datasets: CIFARIO-DVS (Li et al. 2017), DVS128 Gesture (Amir al. (017). and N- 1altech101 (Orchard et al.) 2015). CIFAR1O- DVS is split into four 1 1-way 1-shot tasks, DVS128 Gesture into five 1-way 1-shot tasks, and N-Caltech101 into eight 5-way 5-shot incre- mental tasks. We adopt Spiking VGG variants (5, 9, 11), Spiking Resnet variants (18, 19, 20), and Spikingformer. Further details can be found in Appendix G
OJ CIMESSO
Figure 3: Accuracy curves in each session.
Realistic Implementation. All experiments are conducted on a mobile platform NVIDIA Jetson AGX Orin (NVIDIA) 2025). which features 12-core Arm Cortex-A78AE processor, supporting 64-bit Armv8.2 architecture, with 60W peak power and idle power below 15W.
Training Details. Our implementation for 300 epochs on CIFAR- 100 and MinilmageNet with a batch size 128, and for 100 epochs on CIFAR 10-DVS with batch size of 32, on DVS128 Gesture and N-Caltech101 with smaller batch size of 8. The learning rate, 3, 0, b, and 5 are set to 0.001, 1.2, 0.001, 5, and 0.5, respectively. Each experiment is repeated three times with different seeds.
PyTorch with Adam optimizer. Models trained
6.2 COMPARISON WITH STATE-OF-THE-ART METHODS
Baselines. We select nine FSCIL methods in
Table 2: Comparative results on neuromorphic datasets.
their SNN version, i.e., CEC (Zhang et al.
2021.. FACT (Zhou et al. 2022). S3C (Kalla
& Biswas 2022). BIDIST (Zhao et al., (202)). SAVC (Song et al. 2023). TEEN (Wang et al.
Method
Aag((5)
(R
WARP
30.9210 7.
12.75 1..
2023).
WARP (Kim et al. 2023), CLOSER (Oh
CIAARI-DVSS
40.37.1.00 0.02
34 10..0010.77 79
SAFA-SNN
47.5620.94
36.9620.0
ct al. 2024), and FSCIL-ASP (Liu et al. 2024a), baselines. We adopt an SNN-based
35.41:11.7
13. 13.022
DVS128 Gesture
TEEN
83.13+0..0
74.31
CIL method, ALADE-SNN (Ni et al., 2025). and two SNN training methods (Kim al. CacluuteFFCILL
SAFA-SNN
86.74.0.00
77.91
WARP
25.34
14.40
2022 Meng et al. 2023) ) to evaluate FSCIL formance. See Appendix H for details.
N-Caltech101
34.2510
27. 27.861.
28.20.63
39.6920.67
8

---

Published as a conference paper at ICLR 2026
Table Comparative results with SNN-based training methods for on-device FSCIL.
Time Step
Backbune
(M)
(%))
Anss((R)
Auve (9)
Early Dird
CIFAR- 100
Spiking VGG-5
221.35
24.20
41.21 21.20
50.87
Kim al. 2026
CIFAR- 100
Spiking VGG-9
106.87
22.25=0.a6
39.92+1.07
49.30+1.40
SLTT Meng et al. 2023
Min- Image Net
CIFAR- 100
Spiking VGG-9
Spiking VGG-5
106.87
32.85
25.17=1.00
18.93=0.a1
32.58
41.63 03.63
40.45:40.44
51.12+00.28
CIFARIO.DV
Spiking VGG-9
262.87
10.87
26.60
34.46.0.85
CIFAR-
Spiking VGG-5
27.32
46.47
56.72.0
SAFA-SNN
Mini- Net CIPARIÓ-DVS
Spiking VGG-9
22.63
24.67 +0.55
49.25 24
60.98 1.98-=0.o
Spiking VGG-9
262.63
14.47 +0
35.6552.49
41.67 =0
Accuracy. We test the Top-1 I aceuracy on all seen tasks 0, 1... after the session Alast means our relative improvement in the last session. Harmonic accuracy measures the balance between base and novel class performance after each session 8, i.c., An 2x.A,, X.A.. where As, denotes test ac- An+.An
curacy in base session, and An the average accuracy over sessions & > As reported Table [] SAFA-SNN surpasses the second-best approach by 4.01 % for improvement in the final session and boosts the average performance by 6.07% on Mini-ImageNet. The performance curves presented in Figure3 that SAFA-SNN achieves state-of-the art performance across CIFAR-100 and Mini- ImageNet, respectively. Table 2 shows accuracy on average and the last session compared with WARP and TEEN three neuromorphic datasets, verifying its generalization and robustness per- formance. We find baseline WARP has the worst performance, indicating that the ability of space compaction obtain effective parameter presentation is totally limited, far inferior to our neuronal dynamics and subspace projection. Table shoows that our method outperforms SNN training meth- ods in different settings. This confirms the effectiveness and potential of our method on the SNN deployment. Additional results more models, comparison with soft subnetwork- FSCIL methods, and metric An, are provided Appendix
1.2 and respectively.
Spliing Vide
Spliing Rachet20
10 V901
Figure 4: Actual energy cost in training.
Figure 5: Average training time.
Energy consumption. The measurement of actual energy consumption in training uses the built- in sensor values (c.g., GPU, I/O) between the RAM and storage on the Jetson Orin AGX de- vice (NVIDIA 2025). which is gauged by multiplying power (W) by time. We put the training energy consumption in Figure It can be seen that SAFA-SNN exhibits notably lower energy con- sumption than baselines. Following Yao al. (2023), we present the theoretical inference energy consumption of different ANNs and SNNs in Appendix P with detailed formulas and results.
Time efficiency. 5 compares the time efficiency of SAFA-SNN and baselines with Spiking VGGs. Our neuronal dynamics, combined with Z00 and prototype subspace projection, reduce unnecessary training time without modifying synaptic weights or adjusting parameter spaces, out- performing baselines in both efficiency and effectiveness. (Described in detail in Appendix K
6.3 ABLATION STUDY
We conduct an ablation study to analyze the importance of each compo- nent in SAFA-SNN: Sparsity-Aware neuronal dynamics (SA), Zeroth-Order Optimization (ZOO), and subspace projection of prototypes (SP). We report incremental performance curves on CIFAR- 100 and MinilmageNet with time step 4 and Spiking VGG-9, as shown in Figure 6 We can infer that SA has the worst performance, since it does not consider possible adjustments in feature space, and we view as the baseline. When equipped with SP, shows significant performance gains as the model adapts extract more informative features from base prototypes. We then use ZOO for gradient estimation, as shown in the highest curve, which corresponds to our full SAFA-SNN. Ablations verify that every component in SAFA-SNN boosts FSCIL performance.
Ablation on SAFA-SNN.

---

Published as a conference paper at ICLR 2026
Accervay th
CHARI6S
Mishlmaggnnt
Figure 6: Ablation results of SAFA-SNN.
Figure 7: All-session accuracy of variant shots.
Effects on different N-way-K-shots. We assess shot count impact accuracy by varying 1,2,5,0 on DVSI28 Gesture and CIFARIO-DYS in Figure (Analysis in Appendix D
Hypperpaaammetess Analysis. Results on the effects of key parameters (i.e., 3, 7, b, 6, 7 and X) and time step T, are provided in the Appendix and respectively.
Sparsity-Accurent trade-off Analysis. The sparsity remains up to 80% even when setting T 2,3, on different datasets through the training process, indicating balance in spiking sparsity and accuracy, showing potential computation efficiency (Results in Appendix 0
6.4 MORE COMPARISON ANALYSIS
Comparison with surrogate gradients. ZOO is class of gradient-free optimization methods that approximate true gradients using only function evaluations. Unlike surrogate gradients, which replace the non-differentiable spike gradient with limited-width approximation, ZOO addresses stochastic optimization problem for the non-differentiable as, in non-convex setting. We compare 30, the performance of ZOO with six surrogate gradients, with results and analysis in Appendix P.T
methods. To achieve comprehensive evaluation, in this part, present comparisons with typical projection-bare based methods using ResNet backbones in Ap- pendix P.2 Specifically, the comparison methods include WARP (Kim et al.] 2023) and Subspace- reg [Akyürek et al. 2022). The results show that SAFA-SNN consistently outperforms other sub- space regulation methods in most cases.
Comparison with projection-
Comparison results with ANN-based PEFT methods for FSCIL. While ANN-based FSCIL methods can achieve slightly higher accuracy, SNNs remain highly competitive and provide sig- nificant advantages in efficiency and deployability. The observed performance gap is primarily due to the fact that the PEFT backbones of ANN methods are pretrained on ImageNet-IK whereas our lightweight, spike-driven transformer is specifically designed for on-device FSCIL. Ap- pendix P.3 we compare against three categories of PEFT-based recent methods: ASP (Liu et al.) 2024a), CoACT (Roy et al. 2024), and PriViLege (Park al. 2024).
7 DISCUSSION
A potential limitation is the performance degradation in deeper networks, such as Spiking ResNet- 34, caused by the failure of identity mapping. This limitation can be mitigated by employing SEW- style residual connections. In addition, assuming fixed number of classes per session oversimplifies the dynamics of real-world data streams. Future work will address imbalanced way-shot settings to better reflect practical on-device FSCIL. Analysis on base and new classes, visualization, and further discussion are provided in Appendix Q and
8 CONCLUSION
This paper focuses few-shot class-incremental learning with SNN for on-device scenarios. We proposed SAFA-SNN by incorporating sparsity- aware neuronal dynamics, zeroth-order optimiza- tion for non-differential spike, and subspace projection for updating new-class prototypes. Exten- sive experiments on benchmark and neuromorphic datasets indicate that SAFA-SNN outperforms existing FSCIL methods in both performance and energy efficiency with realistic implementation.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
The work of this paper is supported by the National Key Research and Development Pro- gram of China under Grant No.2022YFB450100,, the National Natural Science Foundation of China under Grant No.62502443, No.62125206. the National Key R&D Program of China No. 2025YFG0100700, and the Zhejiang Provincial Natural Science Foundation of China under Grant No. LD24F020001.
REFERENCES
Afra Feyza Akyürek, Ekin Akyürek, Derry Tanti Wijaya, and Jacob Andreas. Subspace regularizers for few-shot class incremental learning. In International Conference on Learning Representations 2022. International Conference on Learning Representations (ICLR), 2022.
Arnon Amir, Brian Taba, David Berg, Timothy Melano, Jeffrey McKinstry, Carmelo Di Nolfo, Tapan Nayak, Alexander Andreopoulos, Guillaume Garreau, Marcela Mendoza, et al. A low power, fully event-based gesture recognition system. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 7243-7252, 2017.
Srinivas Anumasa, Bhaskar Mukhoty, Velibor Bojkovic, Giulia De Masi, Huan Xiong, and Bin Gu. Enhancing training of spiking neural network with stochastic latency. In AAAI, 2024.
Albert S Berahas, Liyuan Cao, Krzysztof Choromanski, and Katya Scheinberg. A theoretical and empirical comparison of gradient approximations in derivative-fre optimization. Foundations of Computational Mathematics, 22(2):507-566 2022.
Yanqi Chen, Zhengyu Ma, Wei Fang, Xiawu Zheng, Zhaofei Yu, and Yonghong Tian. A unified framework for soft threshold pruning. arXiv e-prints, pp. arXiv-2302, 2023.
Xiang Cheng. Yunzhe Hao, Jiaming Xu, and Bo Xu. Lisnn: Improving spiking neural networks with lateral interactions for robust object recognition. In IJCAI, pp. 1519-1525. Yokohama, 2020.
Lei Deng, Yujie Wu, Xing Hu, Ling Liang, Yufei Ding, Guoqi Li, Guangshe Zhao, Peng Li, and Yuan Xie. Rethinking the performance comparison between snns and anns. Neural networks,
121:294-307, 2020.
Shikuang Deng. Yuhang Li, Shanghang Zhang, and Shi Gu. Temporal efficient training of spiking neural network via gradient re- re-weighting, In International Conference on Learning Representa- tions, 2022.
Shuiguang Deng, Di Yu, Changze Lv, Xin Du, Linshan Jiang, Xiaofan Zhao, Wentao Tong, Xiaoqing Zheng, Weijia Fang, Peng Zhao, et al. Edge intelligence with spiking neural networks. arXiv preprint arXiv:2507. 1406,9 2025.
Jianchuan Ding, Bo Dong. Felix Heide, Yufei Ding, Yunduo Zhou, Baocai Yin, and Xin Yang. Biologically inspired dynamic thresholds for spiking neural networks. Advances in neural infor- mation processing systems, 35:6090-6103, 2022.
Wei Fang, Zhaofei Yu, Yanqi Chen, Tiejun Huang, Timothée Masquelier, and Yonghong Tian. Deep residual learning in spiking neural networks. Advances in neural information processing systems, 14:21056-21069 2021.
Li Fei-Fei, Rob Fergus, and Pietro Perona. Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. Computer Vision and Pattern Recognition Workshop, 2004.
David Freedman, Maximilian Riesenhuber, Tomaso Poggio, and Earl K Miller. Visual categoriza- tion and the primate prefrontal cortex: neurophysiology and behavior. Journal of neurophysiol- ogy, 88(2):99-9441 2002.
Robert French. Catastrophic forgetting in connectionist networks. Trends in cognitive sciences, 3(4):128-135 1999.
11

---

Published as a conference paper at ICLR 2026
Saeed Ghadimi, Guanghui Lan, and Hongchao Zhang. Mini-batch stochastic approximation meth- ods for nonconvex stochastic composite optimization. Mathematical Programming, 155(1):267- 305,
Chen Gong, Zhenzhe Zheng, Fan Wu, Xiaofeng Jia, and Guihai Chen. Delta: A cloud-assisted data enrichment framework for on-device continual learning. In Proceedings of the 30th Annual International Conference on Mobile Computing and Networking, pp. 1408-1422 2024.
Zecheng Hao, Xinyu Shi, Yujia Liu, Zhaofei Yu, and Tiejun Huang. Lm-ht snn: Enhancing the performance of snn to ann counterpart through learnable multi-hierarchicr threshold model. Ad- vances Neural Information Processing Systems, 17:101905-101927, 2024.
Elad Hazan et al. Introduction to online convex optimization. Foundations and Trends@ in Opti- mization, 2(3-4):157-325, 2016.
Luca Herranz-Celotti and Jean Rouat. Stabilizing spiking neuron training. arXiv preprint arXiv:2202.000282 2022.
Mark Horowitz. 1.1 computing's energy problem (and what we can do about it). In 2014 IEEE international solid-state circuits conference digest of technical papers (ISSCC), pp. 10-14. IEEE, 2014.
Dexuan Huo, Jilin Zhang, Xinyu Dai, Jian Zhang, Chunqi Qian, Kea-Tiong Tang, and Hong Chen. Anp-g: A 28-nm 1.04-pj/sop sub-mm 2 asynchronous hybrid neural network olfactory processor enabling few-shot class-incremental on-chip learning. IEEE Journal of Solid-State Circuits, 2025.
Sashank Reddi, Suvrit Sra, Barnaass Poczos, and Alexander Smola. Proximal stochastic methods for nonsmooth nonconvex finite-sum optimization. Advances in neural information processing systems, 29, 2016
Jayateja Kalla and Soma Biswas. S3c: Self-supervised stochastic classifiers for few-shot class- incremental learning. In European Conference on Computer Vision, pp. 432 448. Springer, 2022.
Haeyong Kang, Jaehong Yoon, Sultan Rizky Madjid, Sung Ju Hwang, and Chang D Yoo. On the soft-subnetwork for few-shot class incremental learning. In 11th International Conference on Learning Representations, ICLR 2023, 2023.
Hacyong Kang, Jachong Yoon, Sung Ju Hwang, and Chang D Yoo. Continual learning: Forget- free winning subnetworks for video representations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
Do-Yeon Kim, Dong-Jun Han, Jun Seo, and Jackyun Moon. Warping the space: Weight space rotation for class intremeaatt few-shot learning. In The Eleventh International Conference on Learning Representations, 2023.
Youngeun Kim, Yuhang Li, Hyoungseob Park, Yeshwanth Venkatesha, Ruokai Yin, and Priyadarshini Panda. Exploring lottery ticket hypothesis in spiking neural networks. In Euro- pean Conference on Computer Vision. pp. 102-120. Springer. 2022.
Alex Krizhevsky et al. Learning multiple layers of features from tiny images. d, 2009.
Hongmin Li, Hanchao Liu, Xiangyang Ji, Guoqi Li, and Luping Shi. Cifar10-dvs: an event-stream dataset for object classification. Frontiers in neuroscience, 1 1::30, 2017.
Xiao Li, Chenghua Lin, Ruizhe Li, Chaozheng Wang, and Frank Guerin. Latent space factorisa- tion and manipulation via matrix subspace projection. In International conference on machine learning, pp. 5916-5926. PMLR, 2020.
Shuang Lian, Jiangrong Shen, Qianhui Liu, Ziming Wang, Rui Yan, and Huajin Tang. Learnable surrogate gradient for direct training spiking neural networks. In IJCAI, pp. 3002-3010, 2023.
Binghao Liu, Boyu Yang, Lingxi Xie, Ren Wang, Qi Tian, and Qixiang Ye. Learnable distribution calibration for few-shot class incrementeeee learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(10): 1699--16699 2023.
12

---

Published as a conference paper at ICLR 2026
Chenxi Liu, Zhenyi Wang, Tianyi Xiong, Ruibo Chen, Yihan Wu, Junfeng Guo, and Heng Huang. Few-shot class incremental learning with attention-awe aware self-adaptive prompt. In European Con- ference Computer Vision, pp. 1-18. Springer, 2024a.
Jinlu Liu, Liang Song, and Yongqiang Qin. Prototype rectification for few-shot learning. In Euro- pean conference on computer vision, pp. 741-756. Springer, 2020a.
Qianhui Liu, Jiaqi Yan, Malu Zhang, Gang Pan, and Haizhou Li. Lite-snn: Designing lightweight and efficient spiking neural network through spatial-temporal compressive network search and joint optimization. arXiv preprini vrXiv:2401.14655 2024b.
Sijia Liu, Bhavya Kailkhura, Pin- Chen, Paishun Ting, Shiyu Chang, and Lisa Amini. Zeroth- order stochastic variance reduction for nonconvex optimization. Advances in neural information processing systems, 31, 2018.
Sijia Liu, Pin-Yu Chen, Bhavya Kailkhura, Gaoyuan Zhang, Alfred O Hero III. and Pramod K Varshney. A primer on zeroth-order optimization in signal processing and machine learning: Principals, recent advances, and applications. IEEE Signal Processing Magazine, 37(5):43-54, 2020b.
Changze Lv, Yansen Wang, Dongqi Han, Xiaoqing Zheng, Xuanjing Huang, and Dongsheng Li. Efficient and effective time-series forecasting with spiking neural networks. In International Conference on Machine Learning, pp. 33624-33637. PMLR, 2024.
Xinyue Ma, Suyeon Jeong, Minjia Zhang, Di Wang, Jonghyun Choi, and Myeongjae Jeon. Cost- effective on-device continual learning over memory hierarchy with miro. In Proceedings of the 29th Anmual International Conference Mobile Computing and Networking, pp. 1-15, 2023.
Wolfgang Maass. Networks of spiking neurons: the third generation of neural network models. Neural networks, 10(9):1659-1671, 1997.
Pratik Mazumder, Pravendra Singh, and Piyush Rai. Few-shot lifelong learning. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pp. 2337-2345, 2021.
Qingyan Meng, Mingqing Xiao, Shen Yan, Yisen Wang, Zhouchen Lin, and Zhi-Quan Luo. Towards memory-anc time-efficient backpropagation for training spiking neural networks. In Proceedings of the IEEE/CVF international conference computer vision, pp. 6166-6176, 2023.
Bhaskar Mukhoty, Velibor Bojkovic, William de Vazelhes, Xiaohan Zhao Giulia De Masi, Huan Xiong, and Bin Gu. Direct training of snn using local zeroth order method. Advances in Neural Information Processing Systems, 36: 18994-999994,, 2023.
Emre O Neftci, Hesham Mostafa, and Friedemann Zenke. Surrogate gradient learning in spiking neural networks: Bringing the power of gradient-basee optimization to spiking neural networks. IEEE Signal Processing Magazine, 36(6):51-63 2019.
Wenyao Ni, Jiangrong Shen, Qi Xu, and Huajin Tang. Alade-snn: Adaptive logit alignment in dynamically expandable spiking neural networks for class incremental learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 19712-19720, 2025.
NVIDIA. Nvidia jetson orin.
https: www. http:/wwwwwwwit///////ttttt.w.tt
nvvida a cn httm///utoooooussmaaa nes/ orin/ 2025. Accessed: 2025-08-02.
Junghun Oh, Sungyong Baik, and Kyoung Mu Lee. Closer: Towards better representation learning for few-shot innss-inreemnttlll learning. In European Conference on Computer Vision, pp. 18-35. Springer, 2024.
Garrick Orchard, Ajinkya Jayawant, Gregory Cohen, and Nitish Thakor. Converting static image datasets to spiking neuromorphic datasets using saccades. Frontiers in neuroscience, 9:437, 2015.
Keon-Hee Park, Kyungwoo Song, and Gyeong-Moon Park. Pre-trained vision and language trans- formers are shw-shot incremental learners. In Proceedings of the IEEE/CVF Conference on Com- pufer Vision and Pattern Recognition, pp. 23881 18881-22 1-38900, 2024.
13

---

Published as a conference paper at ICLR 2026
Xuerui Qiu, Malu Zhang, Jieyuan Zhang, Wenjie Wei, Honglin Cao, Junsheng Guo, Rui-Jie Zhu, Yimeng Shan, Yang Yang. and Haizhou Li. Quantized spike-driven transformer. arXiv preprint arXiv:2501.13492, 2025.
Leonardo Ravaglia, Manuele Rusci, Davide Nadalini, Alessandro Capotondi, Francesco Conti, and Luca Benini. tinyml platform for on-device continual learning with quantized latent replays. IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 11(4):789-80) 802, 2021.
Shuvendu Roy, Elham Dolatabadi, Arash Afkanpour, and Ali Etemad. Consistency-guided asyn- chronous contrastive tuning for few-shot dass-incremental tuning of foundation models. arXiv
preprint arXiv:2405. 2666552005.6665,, 2024.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma. Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115:211 1522, 2015.
Ali Safa. Continual learning with hebbian plasticity in sparse and predictive coding networks: survey and perspective. Neuromorphic Computing and Engineering, 4(4):042001, 2024.
Guangyuan Shi, Jiaxin Chen, Wenlong Zhang, Li-Ming Zhan, and Xiao-Ming Wu. Overcoming catastrophic forgetting in incremental few-shot learning by finding flat minima. Advances in neural information processing systems, 34:6747-6761, 2021.
Md Maruf Hossain Shuvo, Syed Kamrul Islam, Jianlin Cheng, and Bashir I Morshed. Efficient accel- eration of deep learning inference on resource-constrained edge devices: A review. Proceedings of the IEEE, (1(()):2--1, 2022.
Christian Simon, Piotr Koniusz, Richard Nock, and Mehrtash Harandi. Adaptive subspaces for few-shot learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4136-4145, 2020.
Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. Ad- vances in neural information processing systems, 30, 2017.
Zeyin Song, Yifan Zhao, Yujun Shi, Peixi Peng. Li Yuan, and Yonghong Tian. Learning with fantasy: Semantic-aware virtual contrastive constraint for few-shot class-incremental learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 24183- 24192, 2023.
Hongbo Sun, Jiahuan Zhou, Xiangteng He, Jinglin Xu, and Yuxin Peng. Finefmpl: fine-grained feature mining prompt learning for few-shot class incremental learning. Proceedings of the Thirty-Thine International Joint Conference Artificial Intelligence, pp. 1299-1307, 2024.
Yu-Ming Tang, Yi-Xing Peng, Jingke Meng, and Wei-Shi Zheng. Rethinking few-shot class- incremental learning: Learning from yourself. In European Conference Computer Vision, pp. 108-128. Springer, 2024.
Xiaoyu Tao, Xiaopeng Hong, Xinyuan Chang. Songlin Dong, Xing Wei, and Yihong Gong. Few- shot class-incremental learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 12183-12192, 2020.
Gina Turrigiano. Homeostatic synaptic plasticity: local and global mechanisms for stabilizing ronal function. Cold Spring Harbor perspectives in biology, 4(1):a005736, 2012.
Qi-Wei Wang. Rei Zhou, Yi-Kai Zhang, De-Chuan Zhan, and Han-Jia Ye. Few-shot class- incremental learning via training-free prototype calibration. Advances in Neural Information Processing Systems, 30:10066-5 2023.
Siqi Wang, Tee Hiang Cheng, and Meng- Hiot Lim. Ltmd: learning improvement of spiking
ral networks with leamnall thresholding neurons and moderate dropout. Advances in Neural Information Processing Systems, 35:28350-28362 2022a.
14

---

Published as a conference paper at ICLR 2026
Xuan Wang, Zhong Ji, Xiyao Liu, Yanwei Pang, and Jungong Han. On the approximation risk of few-shot class- caasemnncremettll learning. In European Conference on Computer Vision, pp. 162-178. Springer, 2024.
Zifeng Wang, Zheng Zhan, Yifan Gong, Geng Yuan, Wei Niu, Tong Jian, Bin Ren, Stratis Ioannidis, Yanzhi Wang, and Jennifer Dy. Sparcl: Sparse continual learning on the edge. Advances in Neural Information Processing Systems, 35:20366-2065 20380, 2022b.
Wenjie Wei, Malu Zhang, Hong Qu, Ammar Belatreche, Jian Zhang, and Hong Chen. Temporal- coded spiking neural networks with dynamic firing threshold: Learning with event-driven back- propagation. In Pruceedings of the IEEE/CVF international conference on computer vision, pp- 10552-10562. 2023.
Wenjie Wei, Malu Zhang, Zijian Zhou, Ammar Belatreche, Yimeng Shan, Yu Liang, Honglin Cao, Jieyuan Zhang, and Yang Yang. Qp-snn: Quantized and pruned spiking neural networks. arXiv preprint (rXXi:2502.05005. 2025.
Davis Wertheimer and Bharath Hariharan. Few-shot learning with localization realistic settings. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp- 2019.
Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, and Luping Shi. Spatio backpropagation for
training high-performance spiking neural networks. Frontiers in neuroscience, 12:331, 2018.
Man Yao, Guangshe Zhao, Hengyu Zhang, Yifan Hu, Lei Deng, Yonghong Tian, Bo Xu, and Guoqi Li. Attention spiking neural networks. IEEE transactions on pattern analysis and machine intel- ligence, (113):93339911, 2023.
Di Yu, Xin Du, Linshan Jiang, Wentao Tong, and Shuiguang Deng. Ec-snn: splitting deep spiking neural networks for edge devices. In Proceedings of the Thirty-Thing International Joint Confer- ence on Artificial Intelligence, pp. 5389-5397, 2024a.
Di Yu, Xin Du, Linshan Jiang. Huijing Zhang, and Shuiguang Deng. Exploiting label skewness for spiking neural networks in federated learning. arXiv preprini arX1v:2412.17305, 2024b.
Di Yu, Changze Lv, Xin Du, Linshan Jiang, Wentao Tong, Zhenyu Liao, Xiaoqing Zheng, and Shuiguang Deng. Ecc-snn: Cost-effective edge-cloud collaboration for spiking neural networks. arXiv preprint arXiv:2505.20833 2025.
Friedemann Zenke and Surya Ganguli. Superspike: Supervised learning in multilayer spiking neural networks. Neural computation, 104)111,,- 2018.
Chi Zhang, Nan Song, Guosheng Lin, Yun Zheng, Pan Pan, and Yinghui Xu. Few-shot incremental learning with continually evolved classifiers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 12455-12464, 2021.
Jinghua Zhang, Li Liu, Olli Silvén. Matti Pietikäinen, and Dewen Hu. Few-shot glass-incremental learning for classification and object detection: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(4):2924 2945, 2025.
Linglan Zhao, Jing Lu, Yunlu Xu, Zhanzhan Cheng, Dashan Guo, Yi Niu, and Xiangzhong Fang. Few-shot class-incrementee learning via class-aware bilateral distillation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 11838-11847 2023.
Da-Wes Wei Zhou, Yun Wang, Han-Jia Ye, Liang Ma, Shiliang Pu, and De-Chuan Zhan. Forward compatible sho- caas--incremental learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9046-9056, 2022.
15

---

Published as a conference paper at ICLR 2026
A SPARSITY-AWARE NEURONAL DYNAMICS
First, active neurons facilitate knowledge updating while stable neurons align with base-class repre- sentations to preserve learned knowledge. This creates clear distinction between neurons encoding base classes and those encoding novel classes, as base classes are abundant while novel classes are searce FSCIL. implements a method similar to subnet regularization, but unlike soft subnetwork approaches (Mazumder et al.) 2021]] Kang et al. 2023 2024). which update task-specific subnet for each task, incurring additional computational overhead and introducing extra trainable parame- ters. Adaptive neurons more plastic and can adapt their firing thresholds more rapidly to encode
new-class information. The whole neuronal dynamics can be formulated as:
I = BN EN(Conv(X,;0))
(16)
U, = -U,-1 + It:
(17)
S. HU, - Uch);
(18)
A = 3(1 - -M) + 7M,
(19)
U = Um + +A(re rb). rb).
(20)
Initially, we divide neurons into act as active and stable neurons with mask M. At time step t. I is the cumulative current with the input X, and parameters 8. The membrane potential U, evolves continuously from time step L to time step f. When the membrane potential rises and reaches threshold Ut. it emits a spike S., Then Uh changes by variation of sparsity with regulator A. Sparsity- aware neuronal dynamics ensures that most neurons exhibit minimal threshold changes and maintain their spike firing within a constrained range under varying sparsity, resulting in relatively stable spike representations of base-class knowledge. Meanwhile, a small subset of neurons with broader range of dynamic threshold adaptation in response to sparsity minimizes updates to the ma- jority of synapses, improves incremental learning adaptability, and mitigates catastrophic forgetting.
B
MEAN SQUARE APPROXIMATION ERROR ANALYSIS
The mean square approximation error (MSE) of the gradient estimate Vg(a) with respect to Vg(æ):
E ((() -
= 0(6)) + + 652
(21)
where we use the big O notation to emphasize the dominant factors, i.c., the dimensionality d and the perturbation radius (or smoothing parameter) 8, and the number of samples b that govern the gradient estimation error (Berahas et al.] (022). At the non-differentiable point u = 0. the true gradient does not exist. However, when z is drawn from a symmetric distribution, the estimator unbiased, i.e., its expectation is zero. Importantly, the non-zero contributions of the estimator are concentrated in a neighborhood around u = O, where the function exhibits non-smooth behavior.
The parameter å plays crucial role in gradient estimation (Liu et al. 2020b). A smaller 8 reduces the bias of the estimator but significantly increases its variance, as the finite-difference computation is more sensitive noise. In practice, excessively small ó may cause the function difference to be dominated by stochastic noise, making the estimator unreliable. In contrast, a larger 5 reduces variance through increased smoothing but introduces greater bias.
As the sample size increases, the estimator converges to its expectation according to the law of large numbers. In the limit as b oo, the estimator converges the gradient of the smoothed objective function rather than the true gradient of the original, non-smoothed objective. At differentiable points, the bias diminishes as + while the variance of single-point estimation of order O (++). Through multi-point averaging, the variance is reduced to O b ), indicating that increasing b can effectively compensate for the variance amplification caused by small 6.
C CRITERIA FOR CONVERGENCE ANALYSIS
We provide four assumptions regarding convergence rates on zeroth-order algorithms, which can be applicable to different types of problems according to the previous literature (Liu et al. 2020b).
16

---

Published as a conference paper at ICLR 2026
(1) Convex optimization: For convex function f, convergence is evaluated by B[f(zr) f(x")),
(22)
where IT denotes the point obtained at the final iteration and x" E &argmin f(z denotes an
aEX'
optimal solution of f. with X closed convex set. The expectation accounts for all randomness.
(2) Online convex optimization: In the online setting, performance is measured by the cumulative regret Regrety Hazan al. 2016 for sequence of convex cost functions {ft] as
T f((z)) min Σ f(()))
E
(23)
(3) Unconstrained nonconvex optimization: In unconstrained nonconvex settings, convergence is assessed by the average expected squared gradient norm across 'T iterations:
B [IV)(()))
(24)
(4) Constrained nonconvex optimization: For optimization over a feasible set X, convergence is often measured by the norm of the projected gradient (Ghadimi et 2016 JReddi al. 2016):
(x (f(a)) m)) = - J.. IIx (ar 1/ [
(25)
7)t
where IIx denotes the projection operator that updates a point subject to the constraint E X, and || || measures how far the current iterate 2; is from satisfying first-order optimality within X.
Overall, ZOO is typically applied to unconstrained nonconvex optimization because it does not require gradient information and can handle highly complex, nonconvex functions. Introducing constraints would complicate the gradient estimation and increase bias and variance.
D PROOF OF ZEROTH-ORDER OPTIMIZATION
We provide theoretical proof of the estimation function g(x) according the literature (Mukhoty et al. 2023) as follows.
Definition 1. function 9 R + Rzo is called surrogate function if is even, non-decreasing on (-00,0), 0), and == -_ g(2) dz < oc.
Lemma 1. Let be a distribution such that
[. 2 1etttx A(2) dz < cc.
(26)
Then the expectation 2s> (w: z, 5)] defines a surrogate function.
Theorem 1. Let p be probability distribution with probability density function (PDF) p(t) such that Jo to p())dt < 3o and f + p(t)dt oc. Define the transformed distribution À with PDF
[ x²)(t) dt,
((2)
141
(27)
where e is a normalization constant ensuring 1_ Å(z)dz 1. Then d
Ex-plg 5)] =
du 1 + 8z)].
(28)
where h is an appropriately defined surrogate function.
The standard normal PDF is
cxp
2²
so that
V2n
x2
u²
Egep ( z, 2)
exp
dz
exp
(29)
00 26 2
2
8m
28
17

---

Published as a conference paper at ICLR 2026
Theorem 2. Let h(u) be surrogate function and define
82
c:= =2
h'(2))z < 00, p(z) =
11 -===aa((z0)) 2> 0.
(30)
za
cza
Then p(z) is a valid PDF and
1R22pl9 (u; 2, = h(u)),
(31)
with derivative
(u) 1²a (exp(=ku))( exp( 1 ku))
(32)
(1
ku))s
Consider the Sigmoid surrogate function, where the differentiable Sigmoid (Zenke & Ganguli approximates the Heaviside function. The corresponding surrogate gradient is defined as
2018
d h(u) == du 1 + exp(-ku)
k exp(
(33)
(1 +
It is easy to verify that h(u) satisfies the properties of a surrogate function: it is even, non-decreasing on (-o0, 0), and integrable over R with I h(u) du = < oc.
Applying Theorem
2 the scaling constant c can be computed as
h'(t)) dt 28 2
exp(-kōtt(( - exp(-1:8t)
82
c 282
dt
(34)
t(1 + exp(-kót))
a
where a
The corresponding probability density function
0.4262
82 (1()) a² exp(-k52)) exp(-kō2)
p(z)
(35)
2
2(1 + exx((xxkkkz)
E SOURCES OF ERROR FOR ZEROTH-ORDER OPTIMIZATION
We employ a two-point ZOO estimator, which queries the model output under small symmetric perturbations and yields significantly more accurate gradient estimate than one-point approxima- tions (Liu et al., 20206).
We first analyze the gradient approximation error o00, denoted by (19(x) [ where (xr)
the true objective function and ( its exact gradient. According to Berahas et al. (2022) the sources of error in approximating the gradient can be summarized into two main folds:
(1) The discrepancy between the true gradient (2) and the gradient of the Gaussian-smoothed surrogate F(x), which constructed based on the noisy observations of
(2) The error introduced when estimating V F through a finite number of sample averages.
Derivative-free optimization often proceeds by approximating the true gradient ( with esti- mate gzx) calculated from noisy function evaluations, which is then used in a gradient-baa based update. By adding and subtracting (zz), the gradient approximation error can be decomposed
l(g(z) [= (VF(2) - v2(x)) (g(x) VF(x))I (x) v4 (l9(z) [=))]]
(36)
Bound on the first term |V(()) (=))) Assume that the noise in the function evaluations e(x) bounded for all x R", and that has L-Lipschitz continuous gradients. Suppose the function f' is an approximation of t with uniformly bounded error, i.e., there exists cf 2 0 such that
If(z) (z)) = |e(=)) < Ee
Vo R.
(37)
has L-Lipschitz continuous gradients, then
[VF(x) v < VoL8+ vbg,
(38)
18

---

Published as a conference paper at ICLR 2026
where 5 is the sampling radius and b is the sample number. If d has M M-Lipschitz continuous Hessians, then
IVF(t)-Vee)) S nM + vber
(39)
Bound the second term l1((x) - V= This emorr arises from the sample-average approxi- mation. Assume that & has L-Lipschitz continuous gradients and its Hessian is M-Lipschitz con- tinuous. Then the Caussian-mmoothed function F inherits the corresponding smoothness proper- ties. Let (xx) denote the two-point estimator computed using N independent Gaussian directions u4 (0, , 1). The covariance of g(z) satisfies
Var(g(=)) 1 N BuwNoon
(
f(x du) f(x - óu) ((z=bu)) )
2
uns
VF(x)VF(=)". (40)
28
Using the result from Berahas al. 2022). when g(x) is computed using the two-point formula, for all E RR, Var(g(æ)) 3 k(x)1, where
3 <(x) N
&f
(n + 2) 4)(n
(41)
36
82
Table 4: Comparison with FSCIL baselines on static dataset CIFAR-100 with 5way-5shot incremen- tal learning setting and Spiking VGG-9.
Method
Accuracy in each session (28) f
Aura
2
FACT
69.52
61.46
57.83
54.53
51.35
18.64
46.20
44.45
42.61
52.97
+5.48
S3C
61.72
31.60
16.00
15.10
11.00
19.40
18.00
25.20
25.00
21.59
+25.09
BIDIST
57.69
+7.73
SAVC
41.75
38.54
35.79
33.40
31.31
23.47
27.83
26.37
25.05
32.17
12.04
TEEN
09.87
63.20
59.49
55.67
52.79
50.11
18.12
16.33
44.51
59.34
+3.58
CLOSER
55.22
52.55
49.30
46.41
44.23
42.06
40.23
39.67
37.21
45.10
10.88
ASP
38.05
46.05
33.29
27.76
27.062
27.48
21.37
21.06
16.04
31.01
+33206
70.91
56.59
Table 5: Comparison with FSCIL baselines on static dataset CIFAR- with 5way-5shot incremen- tal learning setting and Spiking VGG-11.
Method
Accurucy
Aug
A
2
3
4
6
8
CBC
64.37
59.85
55.89
52.04
43.88
46.37
43.92
41.72
39.60
50.63
+7.30
57.48
53.54
50.05
47.0K
42.13
30.92
1-8.88
S3C
16.28
24.80
14.40
16.21
13:60
15.50
31.00
22. 22.10
18.00
20:28
+28 50
SAVC
62.95
58.11
53.96
50.36
47.21
44.14
41.97
39.76
37.77
45.39
+9.13
64.30
59.79
55.73
43.68
7.56
WARP
02:28
57.95
53.99
50.41
47:25
44.69
42.34
40.20
25
48
+8.62
CLOSER
07.87
63.52
39.57
56.
52.83
49.89
47.54
15.13
13.58
51.02
+3.52
76.95
49.16
46.90
Table 6: Comparison with FSCIL baselines on static dataset CIFAR- 100 with 5way-5shot incremen- tal learning setting and Spiking ResNet-20.
Method
Accurncy
Ang
Arue
0
2
3
4
5
6
7
8
CBC
12:45
12.22
11:27
10.35
10.14
8.53
8.15
4-38.75
FACT
61.03
56.51
49.77
47.31
44.51
12.53
10.86
09
47.13
+7.81
S3C
09.15
26.40
22. 80
22.20
22.50
22.00
25.20
33.10
30.1 00
27.38
+16.5
TEEN
65.68
57.79
54.64
51.07
48.30
45.53
43.60
42.00
40.25
49.21
6.65
WARP
36.85
34.39
31.36
29.64
27.5
26540
25.25
23.93
31. 64
+-22.97
CLOSER
56.02
53.43
50.33
47.21
44.00
42.48
40.44
38.88
37.10
45.61
+-0.80
SAFA-SNN
76.95
71.20
66.00
61.81
46.90
58.12
19

---

Published as a conference paper at ICLR 2026
Table Comparison with FSCIL baselines on static dataset CIFAR-100 with 5way-5shot incremen- tal learning setting and Spiking ResNet-19 19,
Method
Accuracy in each session (1)
A
FACT
67.67
62.17
36.03
54.41
51.23
48.64
46.17
44.14
12.09
53.28
+4.17
TEEN
66.71
61.71
37.32
53.69
30.35
17.67
43.
13.11
11.03
52.
+5.23
WARP
66.58
6.12
2.94
3.47
2.46
1.91
1.87
1,92
1.30
10.83
+44.96
SAFA-SNN
73.98
68.74
63.99
59.99
58.38
53.49
51.03
48.60
46.26
56.35
Table 8: Comparison with FSCIL baselines on static dataset CIFAR-100 with 5way-5shot incremen- tal learning setting and Spikingformer.
Accuracy session (%
Method
Asyg
Aiset
0
6
7
TEEN
68.37
52.42
46.67
45.43
42.59
40.08
37.86
35.36
34.07
45.37
+22.48
WARP
63.30
6.45
4.04
4.20
2.48
3.34
2.36
2.35
2.50
10.38
+33.96
CLOSER
23.53
21.86
20.24
18.87
17.95
16.92
13.97
15.20
14.47
18.89
+22.05
SAFA-SNN
62.65
52.09
48.90
46.01
43.65
41.34
39.70 70
38. 25
36.55
45 68
F SUBSPACE PROJECTION
Prototypical networks are a few-shot learning framework designed recognize novel classes using only a small number of labeled examples. The model uses shared feature extractor to embed all input images into a common feature space. For each class, the embeddings of its support samples averaged to form prototype vector that represents the class in this space. Classification is per- formed by measuring the distance between query embeddings and class prototypes, typically using the Euclidean (L2) distance. Each query sample is assigned to the class whose prototype is clos- est in the embedding space (Wertheimer & Hariharan 2019). As classification in prototype-base networks non-parametric,, training primarily focuses optimizing the feature extractor. The model is trained task-wise, consisting of small N-way K-shot, enabling it to produce discriminative prototypes from limited examples. Under such a few-shot constraint, learning complex parametric decision functions is inherently difficult. In SAFA-SNN, we employ a simple yet effective prototype update strategy based on subspace projection, which isolates attribute-relevant information without introducing auxiliary networks or additional weighted loss terms (Li et al. 2020
G DATASETS AND SPIKING ARCHITECTURE DETAILS
G.1 DATASETS
The following static datasets for classification are commonly used FSCIL.
CIFAR-100 (Krizhevsky et al.
2009): It contains 100 classes, with 500 training images and 100 testing images per class. Each the 600 RGB images per class has a size of 32 x 32.
Mini-ImageNNer (Russakovsky et al. 2015): contains 60000 RGB images of size 84 84 pixels, sampled from ImageNet-IK.
Neuromorphic datasets exhibit sparse features, typically obtained from event-based simulators or converted from frame-based datasets.
CIFARIO-DVS (Li et al. 2017): A neuromorphic dataset derived from the original CI- FAR10, where visual inputs are recorded by a Dynamic Vision Sensor (DVS), capturing changes in pixel intensity asynchronous events instead of static frames. It contains 9, 9,000 training samples and 1,000 test samples.
DVS128 Gesture (Amir al.] 2017): gesture recognition dataset consisting of 11 hand gesture categories collected from 29 individuals under three different lighting conditions.
N-Caltech101 (Orchard et al. 2015): A dataset comprises 8,246 event streams, each 300 milliseconds in duration, recorded by event-based camera as it captured dynamic visual
20

---

Published as a conference paper at ICLR 2026
Table Comparison with FSCIL baselines on static dataset MinilmageNet with 5way-5shot incre- mental learning setting and Spiking VGG-5.
Method
Accuracy in each session (1)
A
Abust
CBC
57.28
52.93
49.98
47.12
44.90
42.76
40.29
38.77
37.5T
45.73
+1.06
FACT
51.51
43.91
11.16
39:27
37.68
35.80
95
32.
31.82
38.97
+6.81
S3C
41.01
12.06
17.43
29.85
24.01
25.87
21.32
30.95
28.50
25.28
+10.13
55.32
38.19
37.49
34.34
+5.22
WARP
50.24
25.84
24.13
22.34
21.00
19.97
18.75
18.35
17.66
24.70
+200.7
CL.OSER
48.07
40.16
38.41
33.30
32.67
+55.6
SAFA-SNN
61.31
53.06
49.92
48.12
16.19
43.88
11.40
39.86
38 63
46 .3
Table 10: Comparison with FSCIL baselines on static dataset MinilmageNeet with 5way-5shot in- cremental learning setting and Spiking ResNet-18.
Accuracy each session
Method
Aurx
Aluo
0
3
5
7
CEC
55.23
45.15
40.64
TEEN
54.53
47.59
14.51
42.19
40.14
38.03
36.06
31.53
33.11
41.41
+14.98
CLOSER
56.93
46.72
42.12
36.47
SAFA-SNN
76.03
70.91
66.1
61.55
58.19
55.22
52.77
50.33
48 09
56 59
inputs of Caltech101 (Fei-Fei et al. 2004) images displayed on an LCD screen. These recordings span 101 object categories, preserving the diversity of the original dataset while incorporating temporal event-based representations.
We provide the implementation of neuromorphic dataset splitting methods for FSCIL in the code.
G.2 SPIKING ARCHITECTURE
We transform three popular ANN architectures, i.e., VGG, ResNet, and Transformers, into their spiking neural network (SNN) equivalents, removing the need for floating-point multiplication. Specifically, we configure the Spikingformer with depth of 2, five tokenizer convolutional blocks, and spike-form dimensionality of 128.
We directly encode spikes with layers of LIF neurons. Each formulated unit S generating spikes can be represented as
S SN((B(CCNV(())))
(42)
where X XXX the input with time, C-NV(-) is the convolution layer, BN(-) the batch normalization layer and (.)(.) is the LIF neuron model.
H DETAILS OF BASELINES DESCRIPTION
We establish several baselines, including methods based on SNNs and ANNs, better evaluate proposed framework. We briefly list the existing SOTA methods from the FSCIL literature that we convert into SNN-based architectures for evaluation.
CEC (Zhang et al.) 2021): A framework with dynamically evolving architectures incre- mentally transforms newly added linear classifiers into a graph-based structure.
FACT (Zhou et al., 2022): framework facilitates forward compatibility by synthesizing virtual prototypes during the base training stage, thereby enabling more flexible adaptation to future tasks.
S3C (Klla & Biswas 2022): A framework expands its stochastic classifiers by progres- sively incorporating four angular representations.
BIDIST (Zhao al. 2023 This method assigns learnable weight W, to each task and employs bilateral distillation between the representations of current and previous tasks, simultaneously accommodating novel concepts and retaining base knowledge.
21

---

Published as a conference paper at ICLR 2026
SAVC (Song et al.
2023) A framework advances base task performance through leveraging semantic-awarr fantasy classes combined with contrastive learning, which enhances feature representation.
TEEN (Wang et al. )023): A prototype-tuning based approach presents a training- pro- totype rectification technique, which effectively improves classification robustness without requiring additional training.
WARP (Kim et al.) 2023y: A framework trains backbone and classifier base tasks, then fine-tunes classifier and subnetworks of 0 with multi-axis rotations for distinct representa- tions.
CLOSER (Oh et al. 2024): A framework strikes an effective balance between transfer-
ability and discriminability by employing a mechanism of compressed feature spreading, thereby improving generalization across tasks.
The following three methods are SNN-based approaches, for which we adapt the data loading strat- egy to the FSCIL setting for evaluation.
Early Bird (Kim et al. p022): A pruning technique for deep SNNs inspired by the Lottery Ticket Hypothesis, extracts sparse subnetworks that retain performance comparable to full dense networks.
SLTT (Meng et al. 2023): An SNN training technique computes error signals indepen- dently without relying information from other time steps, substantially reducing the required memory overhead.
ALADE-SNN (Ni et al. 2025): An SNN-based framework for class-incremental learning incorporates feature extractor structure expansion and adaptive logit alignment to accom- modate new tasks.
The recent advancement FSCIL introduces parameter-efficient prompt-base methods with ViT the backbone, demonstrating the strong capability of representation.
ASP (Liu et al. 2024a): It efficiently fine-tunes small prompts on a frozen backbone, lever- aging task-aware and task-invariant prompts to improve selection during testing.
CoACT (Roy et al. 2024): It enables FSCIL in foundation models by employing asyn- chronous contrastive tuning with LoRA modules and consistency regularization to mitigate catastrophic forgetting.
PriViLege (Park et al. 2024): A framework that leverages large pre-trained vision- vision-language models for FSCIL through pre pre-trained knowledge tuning and specialized distillation losses.
Table 11: Performance comparison under different SEW ResNet architectures.
Method
Architecture
Ajast (%)
Aavg (%)
TEEN
38. 38.64+0.55
48, 48.20
SEW ResNet-18
WARP
35.37+0.37
45.43+4033
SAFA-SNN
39.76 10.761..32
49.95 10.29
TEEN
38.45+0.28
48, 48.1440.29
SEW ResNet-34
WARP
36.35+0.10
46.6340.11
SAFA-SNN
39.12
49.46.2.11
TEEN
37.35
46. 46.74.0.21
SEW ResNet-50
WARP
26.1441.30
34.3641.75
SAFA-SNN
38.9440.41
49.00+0.47
22

---

Published as a conference paper at ICLR 2026
1 MORE COMPARATIVE RESULTS
I.1 COMPARISON OF ALL-SESSION ACCURACY
Comparisons with SOTA methods CIFAR-100 using Spiking VGG-9, VGG-11, Spiking ResNet- 20, Spiking ResNet-19, 19, and Spikingformer are reported in Table 5 ImageNet with Spiking VGG-5, Spiking ResNet-18 are listed in Table and TO We observe that as the layer increases, the performance of Spiking ResNet drops significantly, e.g. the accuracy of Spiking ResNet-34 is much lower than that of Spiking ResNet-18. The SEW residual connections support identity mapping (Fang et al. (021). Specifically, we have conducted experiments on SEW- ResNet, as shown in Table II The results show that the SEW residual block, which reconstructs the residual learning and activation order using an element-wise function, prevents performance degradation as the network depth increases. These results demonstrate that SAFA SAFA-SNN consistently achieves the highest final-session performance, highlighting its robustness and effectiveness.
7 and and on Mini-
Table 12: Comparison with Soft SubNetwork FSCIL baselines on static dataset CIFAR-100 with 5way-5shot incremental learning setting.
Method
Architecture
Accuracy in each session (%)
Awgg
Trainable Param
2
3
5
6
FSLL SoftNet WSN
ResiNet- 18
66.10
55.85
51.71
48.59
45.31
53.25
41.52
79.81
38.16
$7.59
+10.25
Ply Gx m..m
ResNet-18 18
72.82
67.31
63.05
58:39
76.00
13.25
01.06
08.83
46.63
57.57
+1.78
18
75.70
25.20
40.00 00.00
32.30
40.30
63.80
40.50
45.10
37.10
42.54
+11.31
SAFA-SNN
Spiking ResNet-18
77.57
72.32
62.79
58
58.63
50.92
60.97
1.2 COMPARISON WITH SOFT-SUBNETWORK BASED FSCIL METHODS AND SPARSITY-AWARE NEURONAL DYNAMICS
We compare the performance of SAFA-SNN with prior soft subNetwork-based FSCIL works on the CIFAR-100 dataset. including FSLL (Mazumder al. 202I), SoftNet (Kang al. 2023). and WSN (Kang et al.] 2024) in Table 12 Specifically, in the last column, we report the additional trainable parameters required (beyond the backbone model parameters). For FSL.L, the extra pa- rameters come from the session-specific trainable parameters P'St and the rotation prediction net- work RR. SoftNet trains an additional soft mask Maoft to select subnetworks the base session. WSN trains a set of parameters 0, and mooft that regularize the weights of the learned SubNet- works. In SAFA-SNN, the trainable is the intrinsic parameters of the LIF neuron model that are trained with membrane potential, and the mask M is used for initialization and then remains fixed. Consequently, SAFA-SNN introduces no additional trainable parameters while achieving the best performance among all compared methods.
Table 13: Performance Comparison across Sessions (A,, / An).
Session
TEEN
36.05/ 25 00
28.07 18.50
05.00/66607
25.53/ 16.00
24.1011492
15.44/1543
21.57 15.31
24.06/14.55
SAFA-SNN
41.90/ 28.80
34.51 22.30
29.25 18.13
27.13/ 16.55
27.74/17.04
27.30 16.73
26.53/ 16.17
27.14/16.68
A
+5.85/ +3.80
+5.H/+3.30
-2.85 1144
+1.00/ +0.35
+3.64/42.12
+2.56/41..8
+1.96 / /0.86
-3.78/-1.73
I.3 HARMONIC MEAN ACCURACY
Table 13 reports the harmonic mean accuracy and novel class accuracy comparison against the runner-up method on CIFAR-100 with Spiking VGG-9, further emphasizing the consistent per- formance improvements and the effectiveness of SAFA-SNN. These results demonstrate that base classes contain abundant semantic information, enabling well-formed prototypes for novel classes with strong prior knowledge after subspace projection and update.
23

---

Published as a conference paper at ICLR 2026
J THEORETICAL ENERGY CONSUMPTION
According to previous studies (Horowitz, 2014 Yao et al, 2023
Lv et al. 2024), for SNNs, the
theoretical energy consumption of layer I can be calculated as:
((I) = EAC x SOPs(I),
(43)
where SOPs is the number of spike-based accumulate (AC) operations.
For traditional artificial neural networks (ANNs), the theoretical energy consumption required by layer can be estimated by:
bb) = AM X FLOPs(b),
(44)
where FLOPs denotes the number of floating- point muultiply-ad-acccummee (MAC) operations. We
assume that both MAC and AC operations are implemented on 45mm hardware (Yao et al. 2023).
where AM 4.6pJ and EAC 0.9pJ. Note that 1 10 mJ 1012 pJ.
The number of synaptic operations at the layer an SNN is estimated as: SOPs(I) T x Ç FLOPs(!),
(45)
where T' is the number of time steps in the simulation, Ç the firing rate of the input spike trains at layer 1.
Based on these theoretical analysis, we report the theoretical inference energy consumption results with variable structures in Table[14, which indicates that ANNs consume substantially higher energy than SNNs. Moreover, the disparity becomes more pronounced as the model size increases, which inherently limits the ability of ANNs to be implemented on edge devices.
Table 14: Inference Energy Consumption Comparison of ANNs and SNNs.
Model
Energy
Model
Energy (J)
Spiking VGG-5
7916.52
VGG-5
10115.37
Spiking VGG-9
VGG-9
5057.97
Spiking VGG-I1
494.92
VGG-11
632.39
Spiking VGG-13
495.00
VGG-13
5689.97
Spiking VGG-16
495.08
VGG-16
5689.97
K TIME EFFICIENCY ON SAFA-SNN
To further validate the scalability and training efficiency of our method, we conducted additional experiments measuring training time architectures, summarized Table [5] indicate that in deeper SpikingFormer models, SAFA-SNN reduces the training time by approximately 80% compared to baseline methods, demonstrating its efficiency in complex architectures.
complex model architectures. Evaluations larger
Table 15: Comparison of computation time across datasets and architectures.
Dataset
Architecture
TEEN (SNN)
CLOSER (SNN)
SAFA-SNN
Mini- Mini-Imaged ImageNet
Spiking 18
104.96s
118.24s
95.96s
CIFAR-100
Spiking 1e
143.17s
372.57s
47.80s
CIFAR-100
Spiking ResNet-20
93.85s
111.06s
69.34s
CIFAR-100
Spikingformer
233.20s
625.67s
47.80s
L EFFECT ON N-WAY K-SHOT
SAFA-SNN leverages N-way K-shot datasets to estimate prototypes for novel classes. To eval- uate the impact the number shots on accuracy, we fix the incremental way and vary shots 11.2,5.10.15.20.55 DVSI28 Gesture and CIFARIO-DVS, We can infer that with more instances per class, the estimation of prototypes will be more precise, and the performance will correspond- ingly improve.
24

---

Published as a conference paper at ICLR 2026
0.5
16.13
61 31
CN
0.24
$0.09
14.55 55
55.55
6-
20.21
40.02
42
Figure 8: Comparison of different hyper-parameters on 6, b, and (3 for on-device FSCIL.
M HYPER-PARAMETER SENSITIVITY
We report the last-session accuracy on CIFAR-100 varying the 3 in [0.4.0.9.1.2] and y in {0.01, 0.04.0.07). We also change the sampled point number from {5,10,15} and constant 5 from (0.1,,.3,0.5), resulting compared results in Figure Our method is robust to the choices of hyper- parameters, and it achieves the best performance when 5 = 0.5 and b 5, 7 0.01 and 8 = 1.2. The adaptive ratio y varies in 10.1, 0.3, 0.5, 0.7, 0.9) and parameter in (0.01, 0.05, 0.1) ). as shown in Table T6 The results suggest that the performance remains stable across variations of and n)
Table 16: Comparison on various 7) and A.
Accuracy in each session (%)
Auvx
2
6
7
0.05
0.1
76.12
70.06
65.3
60.76
57.50
54.65
51.99
48 96
45.77
59.01
0.05
76.12
70.51
65.44
00.83
57.81
54.72
52.30
49.44
46.63 63
59.31
0.05
77.93
72.69
68,
64.09
60.49
57.35
54.80
52.20
49.
61.96
0.05
0.7
76.12
70.57
65.89
61.71
58.09
55.74
52.33
49.85
47.60
59.77
0.05
0.9
75.18
69.45
61.97
60.84
57.14
54.20
51.74
49.23
47.21
58.88
76.73
70.88
66.41
61.73
59.04
55.67
53.24
50.40
45.40
0.1
76.67
70.59
66.11
61.43
58.51
55.02
52.13
49.52
45.40
59.49
N EFFECTS ON DIFFERENT TIME STEPS
Upon proving the importance of SAFA-SNN modules, we further evaluate the impacts of the in- troduced time steps in SNN. We choose time steps T among 1,2,4,6,8 and 10. We report the per- formance in incremental sessions on CIFAR-100 and Mini-ImageNNt Figure 9 respectively. As illustrated in the figures, the best performance is observed when T is approximately between 4 and 6. Since the larger time step often corresponds to challenges like gradient vanishing or exploding YYu al. 2024a), we suggest T 4 the default in experiment.
T-1 - T-2 4D THE -Q-
T-8 +
T
T-30
00
00
75
75-
10
70
65-
45
40
Sassions
Sessions
(a)
Figure 9: Accuracy in each session on different time steps.
25

---

Published as a conference paper at ICLR 2026
O ACCURACY AND SPARSITY TRADE-OFF
We trained the CIFAR-100 dataset on the original Spiking VGG network with 285, neurons in all layers with dynamic threshold. We found that the training very robust to even extreme values of adaptive ratio, as shown in Figure I0 and [1 The points with a red dot and black border are the start point in each setting, while the points with green dot and black border are the end point. We can see that both the sparsity and accuracy converge at high level, showing the promise in the sparsity-accurant trade-off in SAFA-SNN.
6.7
kep VCGST2 Eng USCOT4
0.4
6.5
0.4
6.2
5.84
0.85
0.06
3.57
3.58
0.6
3.81
0.56
0.93
Sparsiy
Sparsity
Figure 10: Sparsity-Acuuracy MinilmageNet. Figure 11: Sparsity- CIFAR-100.
P MORE EXPERIMENTAL RESULTS
P.I COMPARISON WITH SURROGATE GRADIENT METHODS AND ZEROTII-ORDER OPTIMIZATION
We choose the following surrogate gradients (SG): triangle-like surrogate function (Deng et al. 2022). Piccewise Deceeise-Exponential surrogate function (Neftci 2019),
Le
et al. 2020), @PseudoSpike reports the performance differences when training SNNs with 700 and different SG methods. Com- pared to the SG, ZOO estimates gradients using only function values and avoids computationally expensive high-order derivatives in SNNs (Liu et al. 2018). Due the narrow width of SG, neuron membrane potentials often enter saturation regions where the surrogate derivative is nearly zero, causing gradient vanishing or mismatch and widening the gap from the true gradients (Lian et al.) 2023 Although ZOO inevitably introduces some computational overhead from the randomly gen- erated samples, we verify that using only five samples is adequate to maintain effectiveness in experiments. Consequently, the induced overhead is negligible. Developing better Z0O in SNN to reduce computational overhead in the latent space would likely make SNN training easier, and we plan to explore in future work.
Carranz-Cellote & Rouat 2022). Sigmoid, and ATan gradient. Table 17
Table 17: Comparison z00 with six surrogate gradients on CIFAR-100.
Triangle
Piecewise Exp Piecewise Leaky ReLU
@seudoSppike
Sigmoid
ATan
ZOO
Aust (%%)
aan.32
40.98 52.76
45.30 57.58
41.15-417 54.71=1.00
45.91
45.50 45.59
50.13
A (%)
59.254a.12
58.2440.29
57.92 os
61.75
P.2 COMPARISON WITH PROJECTION RELATED FSCIL METHODS AND PROTOTYPE SUBSPACE PROJECTION
Subspace projection a training- free technique that does not incorporate any trainable parameters. Owing the inference-only prototypes and orthogonalization based subspace projection, SAFA- SNN can strengthen the distinctiveness the scarce new class prototypes. The weight space rota- tion in WARP (Kim et al. 2023) constructs weight space identify important parameters.
26

---

Published as a conference paper at ICLR 2026
Subspace-reg (Akyürek et al. 2022) is subspace regularization scheme, focusing complicated trainable modules to pull weight vectors for new classes to close to the subspace spanned by the weights of existing classes. From Table [8 we infer that SAFA-SNN can achieve not only higher performance than previous projection-based techniques for FSCIL, especially in the last in- cremental session, but also lead negligible time cost, compared to previous works that require heavy training.
Table 18: Comparison of SAFA-SNN subspace projection with prior methods in terms of perfor- mance and run-time cost.
Method
Dataset CIFAR- 100
Architecture
Execution Time
Anvz (%)
Alast (%)
Weight Space Rotation
Spiking VGG-9
196.88ms
28.74
38.94
Subspace Projection
CIFAR-100 100
Spiking VGG-9
0.002ms
62.19
50.39
Subspace-reg
Mini-Iman Mati-ImageNet
Spiking ResNet-18
40.5s
53.04
39.00
Subspace Projection
Mini- Etaieee
Spiking ResNet-18
0.002ms
61.75
50.89
P.3 COMPARISON OF SAFA-SNN WITH ANN-BASED FSCIL METHODS
To further substantiate the advancement of the proposed SAFA-SNN, compare method with parameter-efficien ANN based FSCIL approaches (ASP (Liu et al. 2024a). CoACT (Roy et al.] 2024), PriViLege (Park et al., 2024)) via systematic evaluation of accuracy, peak memory footprint (MB), parameter size (M), and energy consumption (mJ) on CIFAR- 100, as summarized in Table 19 For fair comparison, all methods employ the ViT backbone. ANN-based PEFT methods require nearly 10x more memory, 2 more parameters, and 10x more energy, which severely limits their practicality on resource-constrained devices.
Table 19: Comprehensive Comparison with ANN-based PEFT methods on CIFAR-100.
Method
Backbone
Memory
Param
Energy
Anase (%)
Alast (%)
Auvg (%)
ASP (ANN)
VIT
9721.72
662. 19
1017.06
75.5340.36
11.79+1.04
31.27
CoACT (ANN)
VIT
16449.48
205.63
9226.95
47.2544.38
9.39
15.43
PriViLege (ANN)
VIT
1580.66
328.01
9230.48
67. 67.39.1175
44.9610.76
54.58
SAFA-SNN
SNN VIT
1035.30
11.08
424.34
73.73+0.40
48.7040.67
59.45+0.67
Q ANALYSIS OF PERFORMANCE ON FEW-SHOT NOVEL-CLASS
Inspired by Wang et al. (2023), we analyze the prediction results in detail. We define "Misclassified Base classes Ratio" (MBR) for classes and "Misclassified to most similar New classes Ratio" (MNR) for base classes. As shown in Table 20 MBR is much higher than MNR, indicating that new classes are frequently misclassified as base classes more than base classes new. The average accuracy on seen classes is substantially higher than that on unseen classes. Correspondingly, the "Correctly classified to Base class Number and to New class number" as CBN and CNN, otherwise, the "Mistakenly classified to Base class Number and to New class number" as MBN and MNN. We also report the accuracy on seen classes and unseen classes.
Table 20: Detailed prediction results of MBR (%) and MAR (%) between soft calibration-based method TEEN and SAFA on CIFAR-100.
Method TEEN 649778
CBNT
MBN]
MNNJ CNNT MBR, MARJ. Scent
Unseen
4222
12108
5092
0.704
0.005
62.35
14.95
SAFA-SNN 650935
3065
10180
7020
0.592
0.006
73.57
15.43
The improved novel-class accuracy demonstrates that the proposed projection does not introduce ad- ditional bias; instead, it effectively reduces the tendency of predicting novel classes base classes.
27

---

Published as a conference paper at ICLR 2026
0.232
61M
1.25
52A
133
55K
38a
087
Time Step
58h
Figure 12: Firing rate of adaptive and non-adaptive neurons.
R VISUALIZATION OF FIRING RATE
We visualize the average firing rates in each time step of stable (dashed lines) neurons and adaptive neurons (solid lines) on CIFAR-100 100 dataset in Figure 12 where "S-k-A" and "S-k-N" denote the firing rate of adaptive neurons in the session and non-adaptive neurons in the k th session, re- spectively. It is observed that non-adaptive neurons exhibit higher firing rates than adaptive neurons the same session. This implies that adaptive neurons constrain their activity to remain within threshold limits, potentially reducing their responsiveness to novel classes, whereas non-adaptive neurons sustain stable firing behavior to mitigate the risk of catastrophic forgetting.
DISCUSSION
Our method significantly improves the recognition accuracy new classes in on-device FSCIL sce- narios, where catastrophic forgetting and resource constraints severely limit model performance. By addressing the underperformance new classes, our approach offers new insights into the over- looked challenges of class imbalance and forgetting. To the best of our knowledge, this is the first study in the on-device FSCIL literature to systematically analyze the performance degradation of new classes, highlighting the need prioritize their evaluation in future on-device FSCIL research. In addition, our method may also offer potential heuristic effects on initiating training with sparse models for implementing edge intelligence on neuromorphic hardware.
T THE USE OF LLM
We only use LLM to polish some descriptive sentences and check grammatical errors.
28


Reviews:
[{"review_id": "UGV6yIypA5", "reviewer": "Reviewer_qRkQ", "summary": "The paper presents a novel approach, SAFA-SNN, for on-device few-shot class-incremental learning (FSCIL) utilizing Spiking Neural Networks (SNNs). The main contribution of the work is a sparsity-aware SNN architecture designed to mitigate catastrophic forgetting and improve model adaptation to new classes with very few samples. SAFA-SNN integrates several innovative components: sparsity-aware neuronal dynamics, zeroth-order optimization to handle non-differentiability in SNNs, and subspace projection to enhance discriminability of prototypes in the learning process.", "questions": "1. What is the primary novelty: a new SNN neuron/dynamics model, or the application of existing FSCIL strategies (like prototype/projection methods) within an SNN framework? The abstract currently conflates these.\n2. Given the widespread success and efficiency of surrogate gradient methods for training SNNs, what is the specific disadvantage of surrogates in the FSCIL setting that necessitates the use of ZOO? The computational overhead of ZOO is typically higher; does this not negate some of the on-device efficiency gains?\n3. How does SAFA-SNN compare, in terms of final accuracy and energy consumption, to a strong, non-spiking, parameter-efficient FSCIL method (e.g., based on prompt tuning or adapter modules) running on the same hardware? This is the most critical comparison to establish the value of using SNNs.\n4. Can you provide a more detailed, intuitive explanation of the \"sparsity-aware dynamic threshold\" mechanism? How is sparsity measured and enforced? How does this directly relate to stabilizing old knowledge and integrating new knowledge?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. Tackling FSCIL in an on-device, rehearsal-free setting with SNNs is a timely and important research direction, aligning with the needs of edge computing and energy efficiency.\n2. Reporting actual energy consumption measurements from a Jetson Orin AGX device, rather than just theoretical calculations, adds practical credibility to the energy efficiency claims.", "weaknesses": "1. The abstract and introduction are overly high-level and vague. Key terms like \"sparsity-conditioned neuronal dynamics,\" \"zeroth-order optimization,\" and \"subspace projection\" are mentioned but not intuitively explained.\n2. The paper positions itself as the \"first SNN-based solution towards general on-device FSCIL.\" However, it does not adequately address why an SNN-based approach is fundamentally superior for this problem compared to a highly optimized, parameter-efficient ANN-based FSCIL method. The comparison in Table 1 is against other SNN-converted FSCIL methods. To claim a true advancement, comparisons against state-of-the-art non-spiking (ANN) on-device FSCIL methodsare essential. \n3. The abstract claims the method \"alleviates overfitting to novel classes.\" This is a core challenge in FSCIL, but the mechanism (subspace projection) is not sufficiently motivated or explained in the abstract to justify this claim.\n4. The claim of \"20% lower energy cost\" is presented without immediate context. Is this compared to an ANN baseline or another SNN baseline? This crucial detail is missing from the abstract, making the claim difficult to interpret.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uEYpRYJiyq", "reviewer": "Reviewer_pCz7", "summary": "Addressing the challenge of on-device Few-Shot Class-Incremental Learning (FSCIL), this paper proposes SAFA-SNN, a method based on Spiking Neural Networks (SNNs) which are more energy-efficient than traditional ANNs.  The framework introduces sparsity-conditioned neuronal dynamics, where most neurons remain stable to mitigate catastrophic forgetting. To overcome training difficulties with SNNs, it employs zeroth-order optimization for gradient estimation. Furthermore, SAFA-SNN uses subspace projection during incremental learning to enhance the learning of new classes and avoid overfitting. Experiments demonstrate that SAFA-SNN outperforms baselines, achieving at least 4.01% improvement on Mini-ImageNet and 20% lower energy costs.", "questions": "Please refer to the weakness.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "(+) **Pioneering SNN-Based Solution**: It proposes SAFA-SNN, which is noted as the first SNN-based framework designed to solve the general on-device FSCIL problem, offering a novel, energy-efficient alternative to traditional ANNs.\n\n(+) The SAFA is built around Spiking Neural Networks (SNNs), making it inherently suitable for low-power, hardware-friendly deployment on neuromorphic chips or devices like the Jetson series.\n\n(+) The SAFA-SNN framework effectively tackles three core challenges with specific components:\n\n - Sparsity-Aware Dynamics: Helps mitigate catastrophic forgetting by design.\n\n - Zeroth-Order Optimization: Provides a practical solution to train SNNs despite the non-differentiable nature of spikes.\n\n - Subspace Projection: Enhances the model's ability to learn new classes from few examples without overfitting.", "weaknesses": "**Insufficient Justification for Subspace Projection**: The paper fails to demonstrate the specific advantages of its Subspace Projection method. To prove its effectiveness, a comparative analysis against other projection techniques or feature space regularization methods commonly used in FSCIL is necessary.\n\n**Lack of Comparison to Related Methods**: The novelty of the proposed components is unclear due to missing baseline comparisons:\n\n - Sparsity-Aware Dynamics: This mechanism appears conceptually similar to the subnet [1] or the \"soft-subnetwork\" approaches presented in SoftNet [2, 3]. The paper should include a direct comparison to highlight the unique contributions and performance benefits of its approach.\n\n - Zeroth-Order Optimization: The effectiveness of zeroth-order optimization could be contrasted with other methods.\n\n**References**:\n\n[1] Few-shot lifelong learning\n\n[2] On the Soft-Subnetwork for Few-shot Class Incremental Learning\n\n[3] Continual Learning: Forget-Free Winning Subnetworks for Video Representations", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JgHcQzt2IA", "reviewer": "Reviewer_mLap", "summary": "This paper proposes SAFA-SNN (Sparsity-Aware Feature Alignment), a novel framework designed to improve the efficiency and performance of Spiking Neural Networks (SNNs). The core idea is to leverage sparsity-aware alignment between membrane potentials and firing spikes, thereby improving representation stability under sparse activations. The method integrates feature alignment loss with adaptive threshold tuning to balance accuracy and energy efficiency. Extensive experiments on datasets such as CIFAR10-DVS, DVS-Gesture, and ImageNet-DVS are provided.", "questions": "1. How is the “sparsity-aware weighting” computed dynamically — is it global, per-layer, or per-neuron?\n2. What is the relationship between SAFA and temporal consistency loss?\n3. Does SAFA introduce any inference-time overhead? If not, please clarify whether alignment is training-only.\n4. Can SAFA be combined with spike transformer architectures such as QKFormer or SpikingResformer?\n5. Have the authors evaluated the stability of training curves (loss oscillation, firing rate variance)?", "limitations": "", "rating": 2, "confidence": 5, "soundness": 1, "presentation": 2, "contribution": 1, "strengths": "The paper effectively identifies a key issue in SNN training — misalignment between spike-based representations and underlying continuous features due to sparse and discrete firing dynamics. The motivation is clearly articulated and well supported by empirical visualization. The introduction of a sparsity-aware term that aligns latent representations between spike and potential spaces is interesting. It extends prior surrogate gradient ideas by explicitly modeling alignment errors as optimization objectives.", "weaknesses": "1. Although the “sparsity-aware alignment” terminology is new, conceptually it overlaps with prior ideas such as temporal consistency regularization and membrane potential loss. The authors should clarify in what way SAFA differs fundamentally rather than being another variant of existing potential-based regularization.\n2. The sparsity-aware loss is introduced heuristically without formal derivation. It would be beneficial to provide an analysis connecting the proposed alignment loss to information preservation or stability bounds under sparse activations.\n3. The ablation in Table 5 (p. 8) only evaluates the removal of individual terms, but does not show sensitivity to sparsity thresholds or alignment weights. Additionally, no visual explanation of the “feature alignment” effect is provided beyond one layer.\n4. Since the method claims to be “efficiency-oriented,” a more detailed discussion on energy or FLOPs reduction would strengthen the claim. Currently, results are purely accuracy-based, with no measured power or latency comparison on neuromorphic hardware.\n5. It remains unclear whether SAFA-SNN generalizes well under noise, temporal jitter, or unseen sensor dynamics. These aspects are critical for practical deployment of SNNs in event-based vision systems.\n6. The paper describes the feature alignment mechanism intuitively but does not provide a clear mechanistic link between sparsity-aware alignment and improved gradient flow or spike stability. It remains unclear why aligning spike and potential features under sparsity constraints leads to consistent accuracy gains. A more detailed theoretical or empirical explanation (e.g., gradient norm analysis) would be valuable.\n7. Given that the proposed loss introduces additional terms involving sparsity and alignment, it is important to analyze how this affects training convergence and gradient stability. However, there is no discussion or empirical observation (e.g., training curves or variance analysis) to ensure that the added loss does not destabilize optimization.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Fp0X54yqst", "reviewer": "Reviewer_smss", "summary": "This paper applies spiking neural networks (SNNs) to the few-shot class-incremental learning (FSCIL) scenario and proposes SAFT-SNN, which integrates sparsity-aware neuronal dynamics, zeroth-order optimization, and fast adaptive prototype subspace projection. The author deployed the SNN on a mobile platform and conducted experiments on static and neuromorphic datasets. The results demonstrated that their method outperformed other comparative methods.", "questions": "See weakness.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The experimental results comparing the proposed method with other approaches demonstrate its superior performance and efficiency.", "weaknesses": "1. Why do sparsity-aware neural dynamics allow most neurons to remain stable while activating only a few? The authors should explain the method they propose in more detail.\n2. Although the author deployed the proposed method on a mobile platform, they evaluated theoretical power consumption rather than actual runtime power consumption. Moreover, the results in Figure 5 indicate that its speed does not significantly outperform other methods. This casts doubt on the true efficiency of the proposed method.\n3. The proposed SAFA-SNN is nearly impossible to deploy on event-driven neuromorphic chips because Eq. (14) involves computationally intensive matrix multiplication. Therefore, it appears that the efficiency and power consumption advantages of SNNs cannot be realized.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pIMsM7CdGe", "reviewer": "Reviewer_hwBr", "summary": "This study focuses on the Few-Shot Class-Incremental Learning (FSCIL) problem, aiming to enable edge devices to continuously learn novel classes under realistic constraints of limited data and energy resources. To achieve this, the authors propose the SAFA-SNN framework, which comprises three main components: Sparsity-Aware Neuron Dynamics, Zeroth-Order Optimization, and Subspace Projection for prototype alignment. Notably, all experiments are implemented and rigorously evaluated on real edge devices.", "questions": "Overall, the paper is comprehensive and well-structured; however, I would appreciate clarification on a few points:\n\n1. Does the subspace projection introduce prototype bias toward base classes, potentially impairing performance on novel classes?\n\n2. Is the division between active and stable neurons fixed or adaptively updated during training?\n\n3. Why is the initial mask retained during incremental learning phases? Could this lead to over-saturation of active neurons over time?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The computation energy measurement methodology employed in this study is notably fair and well-justified.\n\n2. This work pioneers the exploration of on-device SNN-based FSCIL, establishing a solid empirical baseline for future research.", "weaknesses": "1. Some equations lack proper punctuation (e.g., Eqs. 14 and 15), and the third section of Table 3 could be better centered for readability.\n\n2. Although the Zeroth-Order Optimization (ZOO) section provides an upper bound on convergence, it lacks quantitative comparison with surrogate backpropagation and detailed analysis of potential error sources.\n\n3. The paper mentions the degradation issue in deeper networks (e.g., ResNet-34) but does not provide a systematic explanation or proposed solution.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)