Event-based backpropagation on the neuromorphic platform SpiNNaker2

Authors: gabriel  béna
timo  wunderlich
mahmoud  akl
bernhard  vogginger
t u dresden
  germany
christian  mayr
hector andres gonzalez
Venue: NeurIPS_cc_2024_Workshop_MLNCP
Published: 2024-09-10 19:57:27

Abstract: Neuromorphic computing aims to replicate the brain's capabilities for energy efficient and parallel information processing, promising a solution to the increasing demand for faster and more efficient computational systems. Efficient training of neural networks on neuromorphic hardware requires the development of training algorithms that retain the sparsity of spike-based communication during training. Here, we report on the first implementation of event-based backpropagation on the SpiNNaker2 neuromorphic hardware platform. We use EventProp, an algorithm for event-based backpropagation in spiking neural networks, to compute exact gradients using sparse communication of error signals between neurons. Our implementation computes multi-layer networks of leaky integrate-and-fire neurons using discretized versions of the differential equations and their adjoints, and uses event packets to transmit spikes and error signals between network layers. We demonstrate a proof-of-concept of batch-parallelized, on-chip training of spiking neural networks using the Yin Yang dataset, and provide an off-chip implementation for efficient prototyping, hyper-parameter search, and hybrid training methods.

Main paper: 
 introduction: 
 Neuromorphic computing seeks to emulate the unparalleled efficiency of biological neural networks by implementing spiking neural networks which use sparse, spike-based communication between neurons. This could enable artificial neuronal systems to process temporal, spike-based data with efficiency similar to biological brains. At the same time, backpropagation proved to be a pivotal method in machine learning, allowing for efficient gradient computation and enabling the recent advances in training non-spiking, artificial neural networks on challenging tasks [8] . This suggests that implementing gradient-based learning algorithms on neuromorphic hardware could enable similar achievements while surpassing traditional hardware in terms of energy efficiency. Such learning algorithms should make use of the temporal sparsity afforded by spike-based processing.Building on previous work on gradient-based learning in spiking neural networks [3] , the Event-Prop algorithm [26] uses event-based backpropagation to compute exact gradients, retaining the advantages of temporal sparsity during network training. In contrast, non-spiking neural networks typically require a dense sampling of neuronal variables for backpropagation which introduces a memory bottleneck, limits network size and increases energy consumption due to the required storage and transfer of dense state variable samples [20, 16, 12] . Harnessing the potential of event-based algorithms such as EventProp requires their implementation on suitable event-based hardware architectures.Gradient-based learning using neuromorphic hardware has been previously implemented using "inthe-loop" approaches, where neuromorphic hardware computes spike times and state variables in an inference phase (the forward pass) while a classical computer computes gradients and implements backpropagation for the computation of surrogate gradients [4, 5] or exact gradients [6, 18] . Previous work implemented the E-prop algorithm [1] on SpiNNaker1 [19] and an FPGA-based prototype of SpiNNaker2 [21] . This algorithm computes surrogate gradients using local eligibility traces and a global error signal, without error backpropagation through network layers, which prevents scalability. Besides SpiNNaker, the GeNN code generation framework [10] has been used to create a GPU-based implementation of EventProp [15] . This manuscript presents the first implementation of event-based backpropagation on SpiNNaker2. SpiNNaker2 is a novel digital neuromorphic hardware platform providing a scalable event-based and asynchronous computational substrate [7] . While our results are obtained using a single SpiNNaker2 chip, the platform can scale to large numbers of interconnected chips (e.g., more than 5 million compute cores in the Dresden SpiNNcloud platform [13] ). 
spinnaker2 details: 
 SpiNNaker2 is a massively parallel compute architecture for event-based computing. It is based on a digital, many-core GALS (Globally Asynchronous Locally Synchronous) configuration, where every chip is composed of 152 ARM-based Processing Elements (PEs) with dedicated accelerators for both neuromorphic (e.g., true random number generators, exponential and logarithmic accelerators) and deep learning applications (e.g., multiply-and-accumulate arrays). The PEs are arranged in a two-dimensional array and communicate via a dedicated high-speed Network-On-Chip (NoC). Communication to a host computer is established via 1 Gbit ethernet (UDP) and 2 GB of DRAM on the board can be accessed via two LPDDR4 interfaces. For scalable, system-wide communication, each chip has a dedicated SpiNNaker2 packet router, containing configurable routing tables, and six links to neighbouring chips. Different packet types with up to 128-bit payload allow for efficient communication between PEs, chips and boards. This will allow for SpiNNaker2 systems with up to 65 000 chips and over 10 million cores [13] . Importantly, these ARM Cortex M4F cores provide flexibility for the implementation of arbitrary neuron dynamics and event-based models (e.g., [23] ).A Python-based software package, PY-SPINNAKER2 [25] , allows user to define network architectures and parameters (py-spinnaker2) and is the main entry point for running experiments on-chip. The Python module interacts with the chip through an intermediate layer implemented in C++ that sends and receives data and loads up PEs with the memory files needed to run simulations. Interactions between host and chip are detailed in Figure 1 .Our implementation comprises four programs running on different PEs of a chip: the first injects input spikes, the second simulates a layer of leaky integrate-and-fire neurons, the third computes a time-to-first-spike loss and the fourth accumulates gradients and computes weight updates.A common regularization method in machine learning is to process a subset of the training batch in parallel: gradients are averaged across a mini-batch [8] . Mini-batch training can achieve better generalization while speeding up training by processing training examples in a given mini-batch in parallel. In the case of training on Spinnaker2, we can take advantage of the inherently parallel nature of the architecture to deploy identical copies of the network on multiple cores. Different copies of a network process different input samples, implementing mini-batch training. Except for the optimisation program, all described programs are duplicated on the chip for each training example processed in parallel.We detail each program's implementation next, and provide pseudo-code for each in 6.4. The architecture and software stack is summarised in Figure 1 . This implementation showcases the flexibility of SpiNNaker2 to implement custom training algorithms on neuromorphic hardware. While our implementations only uses 128 kB of SRAM available on each PE, future work will interface with 2 GB of DRAM on the board to enable larger networks and more complex models. During the backward pass, spikes are distributed in reverse order, and carry state variables representing error signals within the 32 bit payload of event packets. At this point, the network runs in "reverse-time" and follows the adjoint dynamics, with error signals being transmitted at the spike times computed during the forward pass. In this way, the backward pass implements a spiking neural network with graded spikes (representing the error signal) at fixed times, making it suited for an event-based implementation on the neuromorphic platform. The backward pass implements a discretization of the adjoint system [26] , following an "optimize-then-discretize" approach. In this way, the program uses event-based backpropagation to compute a discretized approximation to the exact gradients of the simulated spiking neural network. The complete set of discretized equations is provided in section 6.1.Time-to-First-Spike Loss Program: The loss program receives spikes from the output layer and computes the loss as well as the initial error signals sent to the output layer during the backward pass. At the beginning of the simulation, cores running the loss program are loaded with classification labels. After receiving spikes from the output layers, loss cores compute derivatives of a crossentropy loss based on first spike times as given byEQUATIONwhere t s k is time step of the first spike of the kth neuron, l is the index of the neuron that should fire first and τ 0 , τ 1 and α are hyper-parameters of the loss function. Loss cores compute error signals corresponding to the derivative of eq. ( 1) which are sent back to the output layer at the respective spike time steps using spike payloads.Optimisation Program: After the backward pass, each neuron layer has accumulated gradients. A single PE is then tasked with gathering gradients and optimisation of the weights. The PE running this program gathers gradients using direct memory access (DMA) from PEs processing different samples of a batch and sums them locally, implementing a mini-batch gradient update. The summed gradients are used to update an internal state which implements the Adam optimization algorithm [9] . This state is used to compute a new set of weights which is written back to each PE using DMA. The processing of the next sample is then triggered for all cores using a global interrupt signal, synchronizing the processing elements.We consider a learning task using the Yin Yang dataset [11] . This dataset consists of three classes of points in a two-dimensional space ([0, 1] 2 ), resembling a Yin Yang symbol (Figure 2 ). Importantly, the dataset is not linearly separable and designed to be challenging for linear classifiers without any hidden layers, requiring deep networks for effective classification (a shallow classifier achieves around 64 % accuracy [11] ). We use this dataset due to the current memory limitation of our implementation and the clear performance gap between shallow and deep networks, allowing us to validate that our implementation can train a deep network on non linear-separable data. We encode the dataset using spikes by translating each coordinate into a spike time in the interval from t min = 2 to t max = 27, running the networks for a total of T = 28 timesteps. Following [11] , we add coordinates (1 -x, 1 -y) for each data point (x, y). We also add a bias spike at time t = 0, resulting in a total of 5 input neurons. Spike times are discretized according to the simulation time step, with ambiguous data points being removed (Figure 2 , right plot).In addition to the on-chip implementation of the EventProp algorithm, we developed an off-chip simulation that matches the on-chip simulation and can be used for efficient prototyping, convenient debugging and parameter search. This off-chip implementation takes the form of a custom PyTorch [17] package, and is available at pytorch-eventprop. A quantitative comparison of the on-and offchip simulations is provided in Figure 4 , Figure 5 and shows a minimal mismatch between state variables that is expected due to numerical differences. Future work could leverage the off-chip simulation for a hybrid training approach that trains a base model off-chip and then deploys it to SpiNNaker2 for on-chip adaptation.We use the off-chip simulation to perform a Bayesian hyperparameter optimization using the "Weights and Biases" software package [2] . The resulting parameters are given in Table 2 .We present learning results using the Yin-Yang dataset and network's dimensions [5, 120, 3] . Total training-set size is 5000, processing in parallel mini-batches of size 22 . We train for 40 epochs in total, and we average results over 10 different random seeds (Figure 3 , left). These results demonstrate successful on-chip training using event-based backpropagation on SpiNNaker2 as well as a close match between on-and off-chip simulations after training.In addition to batch-parallelised processing, we report on results where the networks are trained on a single sample at a time, for a single epoch, on a limited training-set of 300 samples (Figure 3 , right). Such a scenario is particularly relevant for on-line learning on neuromorphic hardware, where the network is trained on a continuous stream of data, which would be critical for embedded systems and autonomous agents. Again, the results show a close match between on-chip and off-chip simulations, and satisfying accuracy considering the heavy limitations.We conducted performance analysis comparing GPU-based off-chip and Spinnaker implementations based on processing time and energy consumption for single batch computation. Using 30-millisecond input samples, the GPU-based implementation demonstrated significant speed advantages, achieving approximately 10x acceleration compared to real-time processing. The Spinnaker implementation maintained real-time processing capabilities, successfully processing samples within the 61ms (forward + backward + 1 optimization step) window. Power efficiency measurements revealed substantial differences between the implementations. The Spinnaker implementation exhibited minimal power consumption of under 0.5W, while power measurements for the GPU-based implementation indicated a consumption of 13.5 W for the GPU device alone. It is important to note that this measurement excludes the power consumption of other system components such as the CPU, memory, and peripheral devices, suggesting that the total system power draw would be notably higher. These GPU measurements warrant additional careful interpretation due to their collection on a shared workstation environment where other processes may have influenced power draw. To establish a more rigorous comparison, future work could leverage embedded GPU platforms such as the Jetson Nano, which would provide isolated testing conditions and precise power monitoring capabilities. Such platforms would allow for measurement of both isolated GPU power consumption and total system power draw, potentially offering a more equitable comparison with the Spinnaker implementation's full-system measurements. While our current measurements suggest superior energy efficiency for the Spinnaker implementation, a comprehensive comparison would require controlled testing environments and dedicated power monitoring infrastructure to draw definitive conclusions about the relative energy efficiency of these approaches. The profiling results are shown in  
discussion: 
 This work reports on the first, proof-of-concept implementation of event-based backpropagation on SpiNNaker2. While our results are limited by the memory capacity of each processing element, future work will leverage the 2 GB of DRAM available on the host board to enable larger networks and the processing of more complex data sets. However, scaling the current implementation to larger networks and multi-chip systems will require addressing several challenges. For instance, the discretization used in this implementation of EventProp may impose constraints on the scalability and performance of multi-chip systems. Discretization errors accumulate and likely limit the scalability of this approach, especially for deep networks or networks with long temporal dependencies. This challenge could be addressed by using other neuron models and numerical schemes that allow for scalable event-based neural networks.While our work is based on spiking neurons, its applicability is not confined to this model class. Advances in event-based machine learning, such as the event-based gated recurrent unit [23] , suggest the possibility of extending neuromorphic event-based backpropagation to models that use eventbased processing with graded spikes. Hybrid hardware such as SpiNNaker2 that supports both eventbased communication and the acceleration of conventional deep neural networks could facilitate the integration of dynamic sparsity benefits into broader machine learning applications, leading to orderof-magnitude improvements in energy efficiency compared to traditional computing [14] .Our on-line learning results suggest that event-based backpropagation could be used for on-chip learning in applications where training data arrives continuously. The computational complexity of the backward pass corresponds to that of the forward pass, making it feasible to use EventProp in such a scenario even if the algorithm is not online strictly speaking. Moreover, it would be possible to implement reinforcement learning methods such as policy gradient [24] in an on-line fashion, since EventProp only needs reward signals and not labels. Our framework could use the off-chip simulation to train a base model that is then deployed and fine-tuned on the neuromorphic hardware using data arriving in real time. To achieve efficient and adaptive on-chip fine-tuning across a range of tasks, the off-chip simulation could implement an outer meta-training loop. For example, model agnostic meta learning (MAML) can be used to find an optimal initialisation that ensures quick and effective adaptation when deployed on-chip, and has been shown to be successfully applicable to SNNs [22] . Such a hybrid approach could leverage and combine the advantages of conventional and neuromorphic computing, and enable the deployment of autonomous and adaptive agents on edge-devices.Our work provides a proof-of-concept implementation of event-based backpropagation using Event-Prop on SpiNNaker2. Event-based backpropagation reduces the demand for memory compared to backpropagation-through-time in non-spiking neural networks and therefore allows for larger network sizes given a fixed amount of memory. At the same time, implementing event-based backpropagation on natively event-based computational substrates such as SpiNNaker2 could enable higher energy efficiency due to temporally sparse data transfers between neurons. Realising these advantages will require addressing the challenges outlined above, and gradient-based learning in spiking neural networks remains an active area of research. Our work demonstrates that the flexibility afforded by SpiNNaker2 can be used to implement future advances in event-based training methods for spiking neural networks. 
acknowledgement: 
 This project is partially funded by the EIC Transition under the "SpiNNode" project (grant number 101112987), and by the Federal Ministry of Education and Research of Germany in the programme of "Souverän. Digital. Vernetzt.". Joint project 6G-life, project identification number: 16KISK001K. This work was partially funded by the German Federal Ministry of Education and Reseach (BMBF) within the KI-ASIC project (16ES0996), and by the BMBF and the free state of Saxony within the ScaDS.AI center of excellence for AI research. The authors would like to acknowledge the 2022 Telluride Neuromorphic Cognition Engineering Workshop where this project was initiated. We present here pseudo code for all chip programs to enable readers a clearer idea of the way computation is done on-chip. These should not be taken literally, albeit being close to the actual code they include simplifications for better reading comprehension.  
 
 Appendix:  



Reviews: review: title: title: A proof-of-concept implementation of event-based backpropagation using Event-Prop on SpiNNaker2
main_review: main_review: The submission provides a proof-of-concept implementation of event-based backpropagation using Event-Prop on SpiNNaker2. The effort is important and relevant for MLNCP as it explores neuromorphic computing as an emerging ML paradigm as well as furthers the pursuit of learning algorithms for neuromorphic. The submission is well written and clear with a base result. Overall, I rate it as a good paper that should be accepted. In the nature of the MLNCP workshop's intention to share work-in-progress, the submission does need additional content to be ready for a formal publication. 

A few points of feedback include: 
- Figure 1 is very dense and difficult to follow. It seems like some of the terms and software blocks require a deep understand of SpiNNaker2 functionalitiy. 
- While the work acknowledges a scaling limitation and the desire to leverage a larger 2GB onboard DRAM, why is it not possible to use a larger example now? Do the 148 unused processing elements not have any memory? Could the parallelization be spread to every PE rather than 4?
- Why is the data cleanup (removing data points) necessary? 
- The accuracy comparison between the off-chip and on-chip baseline is useful for this proof-of-concept. But additionally, how fast is the on-chip learning? Given the desire for neuromorphic computing to be able to perform on-chip learning additional performance details will be necessary for publication.
title: title: An impressive fully on-chip training SNN experiment... But how much better than feedforward nets training by backprop is it?
main_review: main_review: **Summary.** The present work demonstrates for the first time *on-chip training* of an SNN on the SpiNNaker2 neuromorphic platform using EventProp, a learning technique which doesn't require the storage of the full inference computational graph but only that of the spiking times, yielding important memory savings. Let's emphasize this is *not* an in-the-loop experiment where only inference is performed on the chip: *everything* -- inference, loss computation, backward pass and weight update -- is performed on-chip. The paper demonstrate "successful" training on the Yin-Yang dataset, a 3 class 2d nonlinear classification task in the sense that i) the model performs much better than a shallow classifier, ii) the on-chip experiment matches its "off-chip" counterpart -- where the chip is emulated on standard hardware -- in terms of forward and adjoint voltage alignment (Fig. 4), and therefore of weight gradient alignment (Fig. 5) and of resulting training performance (Fig. 3).
More precisely, the paper describes the SpiNNaker2 high level architecture (2.1), the four types of programs run on the Processing Elements (input processing, forward pass, backward pass, loss computation with Eq. 1, weight updates, 2.2) and how *mini-batching* is achieved (Fig. 1). Then the dataset is introduced (2.3), the idea of benchmarking the hardware experiment with an "off-chip" experiment (2.4) and final results thereof (3, Fig. 3) -- note that an "online setting" with batch size of 1 and fewer training examples is also presented. Then future developments of the work are discussed (4). 

**Strengths.**

- The paper is extremely clear, easy to follow and the figures are neat.
- The hardware experiment is the first of its kind on SpiNNaker2, where *everything* is done on chip (including gradient computation and parameter optimization).
- The alignment in terms of forward voltages, adjoint voltages and gradients between off-chip and on-chip experiments is impressive.
- The system allows for mini-batching. 

**Weaknesses.**

- I don't understand the relevance of the dataset (static inputs) when using spikes. Why not a *temporal* task?
- When using the full dataset (Fig. 3, left panel), I'm astonished to see a higher variance in resulting performance for the off-chip experiment compared to the on-chip experiment. I would rather have expected the opposite. It would be good to provide some explanation of why this is the case.
- *"Event-based backpropagation reduces the demand for memory compared to backpropagation-through-time in non-spiking neural networks and **therefore allows for larger network sizes given a fixed amount of memory"***. Currently, you are fitting a 5-120-3 net onto the architecture to be trained by EventProp. To further ground the claim about memory savings, it would have been interesting to say which "maximal" architecture size you could have fit onto the *same* chip **if you were using a standard feedforward net trained by standard backprop**. While I do understand why storing the full computational graph of a *convergent* architecture (such as a SNN) would take a lot of memory, is the memory usage of the computational graph of a *feedforward* net so much more important than that of the firing time of the SNN you are training?
- In the light of the previous comment, **I would like to see better benchmarking, on the *same chip*, when training a feedforward net by backprop**.