A nalytic KWS : Towards Exemplar-Free Analytic Class Incremental Learning for Small-footprint Keyword Spotting (2025)

Abstract: Keyword spotting (KWS) offers a vital mechanism to identify spoken commands in voice-enabled systems, where user demands often shift, requiring models to learn new keywords continually over time. However, a major problem is catastrophic forgetting, where models lose their ability to recognize earlier keywords. Although several continual learning methods have proven their usefulness for reducing forgetting, most existing approaches depend on storing and revisiting old data to combat catastrophic forgetting. Though effective, these methods face two practical challenges: 1) privacy risks from keeping user data and 2) large memory and time consumption that limit deployment on small devices. To address these issues, we propose an exemplar-free Analytic Continual Learning (AnalyticKWS) method that updates model parameters without revisiting earlier data. Inspired by efficient learning principles, AnalyticKWS computes a closed-form analytical solution for model updates and requires only a single epoch of adaptation for incoming keywords. AnalyticKWS demands fewer computational resources by avoiding gradient-based updates and does not store old data. By eliminating the need for back-propagation during incremental learning, the model remains lightweight and efficient. As a result, AnalyticKWS meets the challenges mentioned earlier and suits resource-limited settings well. Extensive experiments on various datasets and settings show that AnalyticKWS consistently outperforms existing continual learning methods.

## introduction as a key component of edge intelligence, devices such as robots, autonomous systems, and smart assistants interact naturally with humans through voice @xcite @xcite . spoken keyword spotting (kws) @xcite identifies specific keyword phrases within recorded speech and is essential for edge computing devices. these devices require quick responses, low energy consumption, and high accuracy to meet user demands. cloud-based solutions may not be ideal in these setups because sending private data to a remote server can violate privacy rules, and real-time updates often require immediate on-device adaptation. due to the kws system always being applied in practical real-world scenarios, modern small-footprint kws systems @xcite @xcite based on deep learning often use compact models to balance performance and computational cost. however, these systems face significant challenges as their performance usually drops when encountering new keywords in the target domain. with the increasing demand for voice as the mode for interaction-oriented tasks in embodied ai, it is important to support more personalized applications @xcite , such as smart home devices and in-car assistants. these devices must continuously learn new keywords while respecting user privacy and resource limits. however, re-training a kws model from scratch with new keywords is not only time-consuming, but also resource-intensive. previous work @xcite @xcite addresses this issue through a few-shot finetuning, which adapts a model to target data with minimal samples but suffers from catastrophic forgetting @xcite , where previous knowledge deteriorates. to solve the forgetting issue, continual learning (cl) @xcite integrates new data while retaining previous knowledge. within this framework, class incremental learning (cil) @xcite focuses on adding new classes to the model sequentially, making it especially relevant for kws that involve evolving label sets. recent cil strategies split into rehearsal-based methods that store past examples for future training and exemplar-free approaches that do not keep old data. for rehearsal-based methods, xiao et al. @xcite first suggested choosing examples through a diversity-based approach for kws. based on that, the latest works of peng et al. @xcite further saved model predictions to distill prior knowledge. however, due to the constraints present in real-world, rehearsalbased cil is often not reliable for kws. first, storing past examples risks breaching user privacy. second, it consumes excessive memory, which is not feasible for resource-limited edge devices. although exemplar-free class incremental learning (efcil) methods avoid storing historical data and thus bypass privacy concerns @xcite @xcite , many of these methods still rely on complex optimizers or dynamic network structures. this approach can be unsuitable for edge devices, which lack the computational power for extensive gradient-based updates. hence, we propose a more efficient method that preserves the benefits of efcil but removes the need for complex adaptations, making it more practical for resource-constrained kws systems. as we mentioned, the key challenge in incremental kws is catastrophic forgetting, where new keywords overwrite knowledge of previous ones. existing solutions address this issue but often store prior data, creating privacy risks and high memory use. to avoid these concerns, we propose analyt-ickws , an exemplar-free method that mitigates catastrophic forgetting while eliminating the need for using past examples. drawing on analytic learning @xcite , analytickws uses a recursive least-squares procedure in place of backpropagation, letting it incorporate new knowledge and protect user data. our core contribution is to maintain previous knowledge without retaining retrospective data, thus resolving catastrophic forgetting in a privacy-preserving and resource-efficient manner. we evaluate the proposed analytickws for a wide range of incremental kws task settings to demonstrate its effectiveness. moreover, by processing new keywords in a single forward pass without gradient updates, analytickws has the capability to lower the computational overhead making it ideal for edge devices. the primary contributions of this paper can be summarized as follows: continual learning for speech processing: cl has shown promise in addressing incremental speech processing tasks by enabling systems to adapt to new data while mitigating catastrophic forgetting @xcite @xcite . chen et al. @xcite proposed a hypergradient-based exemplar strategy for dialogue systems, periodically retraining models using selected exemplars. xiao et al. @xcite introduced an unsupervised framework with distillation loss to add new sound classes while maintaining task consistency. cl has also been explored for incremental kws. rk proposed in @xcite first introduced a diversity-based sample mechanism to select representative exemplars. more recently, de-kws @xcite saved model predictions to distill past knowledge beyond exemplars. however, these methods rely on storing exemplars, which creates challenges for memory-and privacy-constrained on-device applications. to this end, we propose constructing a lifelong kws system without storing the previous predictions or data in this work. analytic learning. analytic learning (al) uses least squares (ls) to obtain closed-form solutions, providing an efficient alternative to back propagation. recently, the recursive formulation (e.g., brmp @xcite ) of al brings inspiration to cl. the brmp can stream new samples to update the weight without weakening the impact of previous samples. acil @xcite was the first to apply al to cl by reframing training as a recursive ls procedure, achieving accuracy similar to joint training for linear classifiers. however, our work advances al in the speech domain by proposing the analytickws method, which adopts an exemplar-free strategy. through recursive updates, analytickws preserves knowledge without storing any past data, representing a notable step forward for al-based cl in speech processing. ## task-wise analysis feature expansion regularization acc(% ↑) ✗ ✓ 86.57 ✓(64) ✓ 87.19 ✓(128) ✓ 88.72 ✓(256) ✓ 89.23 ✓(512) ✓ 89.68 ✓(512) ✗ 89.64 fectively balances long-term retention and newclass adaptation. ## ablation study this ablation study compares models with different acoustic feature expansion (afe) sizes and regularization settings as reported in removing regularization at this level slightly decreases accuracy to 89.64%. these findings confirm that combining an expanded feature space with regularization is crucial to maximize accuracy, while models lacking either approach exhibit lower performance. this result demonstrates the effectiveness of our proposed method. ## conclusion in this work, we have introduced a novel exemplarfree analytic cl method, namely analytickws that addresses catastrophic forgetting and protects data privacy by avoiding the storage of historical examples. incorporating a closed-form analytic update, our approach maintains knowledge across multiple tasks and ensures that incremental learning matches the performance of joint training without requiring repeated access to old data. the recursive structure of analytickws grants absolute memorization, allowing it to achieve state-of-theart results in both small-scale and large-phase scenarios. our experiments on various kws benchmarks verify these benefits, highlighting analyt-ickws's potential for practical deployment on resource-limited devices.