  Spiking Neural Networks (SNNs) bear the potential of efficient unsupervised
and continual learning capabilities because of their biological plausibility,
but their complexity still poses a serious research challenge to enable their
energy-efficient design for resource-constrained scenarios (like embedded
systems, IoT-Edge, etc.). We propose SpikeDyn, a comprehensive framework for
energy-efficient SNNs with continual and unsupervised learning capabilities in
dynamic environments, for both the training and inference phases. It is
achieved through the following multiple diverse mechanisms: 1) reduction of
neuronal operations, by replacing the inhibitory neurons with direct lateral
inhibitions; 2) a memory- and energy-constrained SNN model search algorithm
that employs analytical models to estimate the memory footprint and energy
consumption of different candidate SNN models and selects a Pareto-optimal SNN
model; and 3) a lightweight continual and unsupervised learning algorithm that
employs adaptive learning rates, adaptive membrane threshold potential, weight
decay, and reduction of spurious updates. Our experimental results show that,
for a network with 400 excitatory neurons, our SpikeDyn reduces the energy
consumption on average by 51% for training and by 37% for inference, as
compared to the state-of-the-art. Due to the improved learning algorithm,
SpikeDyn provides on avg. 21% accuracy improvement over the state-of-the-art,
for classifying the most recently learned task, and by 8% on average for the
previously learned tasks.
