Spiking neural networks are nature's solution for parallel information processing with high temporal precision at a low metabolic energy cost. To that end , biological neurons integrate inputs as an analog sum and communicate their outputs digitally as spikes, i.e., sparse binary events in time. These architectural principles can be mirrored effectively in analog neuromorphic hardware . Nevertheless, training spiking neural networks with sparse activity on hardware devices remains a major challenge . Primarily this is due to the lack of suitable training methods that take into account device-specific imperfections and operate at the level of individual spikes instead of firing rates. To tackle this issue, we developed a hardware-in-the-loop strategy to train multi-layer spiking networks using surrogate gradients on the analog BrainScales-2 chip. Specifically, we used the hardware to compute the forward pass of the network, while the backward pass was computed in software. We evaluated our approach on downscaled 16x16 versions of the MNIST and the fashion MNIST datasets in which spike latencies encoded pixel intensities. The analog neuromorphic substrate closely matched the performance of equivalently sized networks implemented in software. It is capable of processing 70 k patterns per second with a power consumption of less than 300 mW. Added activity regularization resulted in sparse network activity withabout 20 spikes per input, at little to no reduction in classification performance. Thus, overall, our work demonstrates low-energy spiking network processing on an analog neuromorphic substrate and sets several new benchmarks for hardware systems in terms of classification accuracy, processing speed, and efficiency. Importantly, our work emphasizes the value of hardware-in-the-loop training and paves the way toward energy-efficient information processing on non-von-Neumann architectures .
To rapidly process temporal information at a low metabolic cost , biological neurons integrate inputs as an analog sum but communicate with spikes, binary events in time. Analog neuromorphic hardware uses the same principles to emulate spiking neural networks with exceptional energy-efficiency . Nevertheless, emulating high-performing spiking networks on such hardware remains a significant challenge due to device-mismatch and the lack of efficient training algorithms. Here, we introduce a general in-the-loop learning framework that resolves these issues. Using the BrainScales-2 neuromorphic system, we show that learning self-corrects for device mismatch resulting in competitive spiking network performance on vision and speech benchmarks. Our networks display sparse spiking activity with, on average, far less than one spike per hidden neuron, perform inference at rates of up to 85 k frames/second, and consume less than 200 mW. In summary, our work sets several new benchmarks for low-energy spiking network processing on analog neuromorphic substrates and constitutes an important step toward on-chip learning .
R
Spiking neural networks are nature's solution for parallel information processing with high temporal precision
To rapidly process temporal information
R
energy cost. To that end
cost
R
and communicate their outputs digitally as spikes, i.e., sparse
but communicate with spikes,
R
These architectural principles can be mirrored effectively in analog neuromorphic hardware
Analog neuromorphic hardware uses the same principles to emulate spiking neural networks with exceptional energy-efficiency
R
training spiking neural networks with sparse activity on hardware devices remains a major challenge . Primarily this is due to
emulating high-performing spiking networks on such hardware remains a significant challenge due to device-mismatch and
R
suitable training methods that take into account device-specific imperfections and operate at the level of individual spikes instead of firing rates. To tackle this issue, we developed a hardware-in-the-loop strategy to train multi-layer spiking networks using surrogate gradients on the analog
efficient training algorithms. Here, we introduce a general in-the-loop learning framework that resolves these issues. Using the
R
chip. Specifically, we used the hardware to compute the forward pass of the network, while the backward pass was computed in software. We evaluated our approach on downscaled 16x16 versions of the MNIST and the fashion MNIST datasets in which spike latencies encoded pixel intensities. The analog neuromorphic substrate closely matched the performance of equivalently sized networks implemented in software. It is capable of processing 70 k patterns per second with a power consumption of less than 300 mW. Added activity regularization resulted in sparse network activity withabout 20 spikes per input, at little to no reduction in classification performance. Thus, overall, our work demonstrates
neuromorphic system, we show that learning self-corrects for device mismatch resulting in competitive spiking network performance on vision and speech benchmarks. Our networks display sparse spiking activity with, on average, far less than one spike per hidden neuron, perform inference at rates of up to 85 k frames/second, and consume less than 200 mW. In summary, our work sets several new benchmarks for
R
an analog neuromorphic substrate and sets several new benchmarks for hardware systems in terms of classification accuracy, processing speed, and efficiency. Importantly, our work emphasizes the value of hardware-in-the-loop training and paves the way toward energy-efficient information processing on non-von-Neumann architectures
analog neuromorphic substrates and constitutes an important step toward on-chip learning