SlowGuess/NeurIPS_2023_OCR huggingface dataset:
SparseProp can also be used for univariate neuron models where no analytical solution between spike times is known, e.g., the exponential integrate-and-fire model (EIF)[37, 38]. In this case, the phase transition curve can be calculated numerically before the network simulation. During run time, the phase transition curve and its derivative can be interpolated from the precomputed lookup tables or using Chebyshev polynomials (see appendix B). An example implementation of an EIF network with SparseProp in Julia [25] is available here. To test the numerical accuracy of the solution based on a lookup table or Chebyshev polynomials, we compared the spike times of two identical networks of LIF neurons with identical initial conditions that were simulated either with the analytical
phase transition curve or using the numerically interpolated solution. We used LIF networks for this comparison, because pulse-coupled inhibitory LIF networks are non-chaotic [11, 21, 39, 40, 41]. Thus, numerical errors originating at the level of machine precision are not exponentially amplified unlike in the chaotic pulse-coupled QIF network dynamics [10]. We show in Fig. 4 that the difference in spike time between the two networks remain close to machine precision and the temporal order of the network spikes is not altered, when replacing the analytical phase transition curve by the
Figure 4: Left: Error of individual spike times for SparseProp are close to machine precision. Right: Error in spike index. Note that despite the small errors in the spike times that are close to machine precision, for non-chaotic network dynamics the spike index is still correct.
6 Time-based SparseProp for Heterogeneous Networks
For heterogeneous networks, where there is heterogeneity of external input current  $I_{i}^{\mathrm{ext}}$ , or membrane time constant  $\tau_{i}$ , or threshold voltages  $V_{\mathrm{th}}^{i}$ , every neuron has a different phase velocity  $\omega_{i}$  and a single global phase shift as described in section 2.1 does not suffice.
In that case, we suggest yet another representation. Instead of having a priority queue of phases, instead, we suggest a priority queue of the next unperturbed spike time for all neurons, denoted by  $\mathbf{n}(t_0)$ , as this can take heterogeneity into account. In this case, the algorithm for network simulation still consists of four steps: First finding the next spiking neuron, then updating the time followed by the update of the postsynaptic neurons by the incoming spikes, finally, resetting the spiking neuron to the reset value. This algorithm also has a numerical complexity of  $\mathcal{O}\left(K\log (N)\right)$  per network spike. We provide pseudocode for SparseProp in the time-based representation below:
Algorithm 2 SparseProp: Event-Based Simulation of Heterogeneous Sparse Spiking Network
1: Initialize heterogeneous phase velocity  $\omega_{i}$  based on the neuron model  
2: Initialize unperturbed interspike interval  $T_{i}$   
3: Initialize time to next spike for all neurons  $\mathbf{n}(t_0)$   
4: Initialize global time shift  $\Delta_t = 0$   
5: Heapify  $\mathbf{n}(t_0)$   
6: Perform warm-up of network state  $\mathbf{n}(t_0)$   
7: for  $s = 1 \rightarrow t$  do  
8: Get index and spike time of next spiking neuron:  $j, n_j = \text{peek}(\mathbf{n}(t_s))$   
9: Calculate time increment:  $\mathrm{dt} = n_j(t_s) - \Delta_t$   
10: Update global time shift:  $\Delta_t += \mathrm{dt}$   
11: Update postsynaptic neurons  $i^* : n_{i^*}^+(t_s) = \text{update}((n_{i^*}^-(t_s), \Delta_t)$   
12: Reset spiking neuron:  $n_j(t_{s+1}) = \Delta_t + T_i$   
13: if  $\Delta_t > \Delta_t^{\text{th}}$  then  
14:  $\mathbf{n} = \Delta_t$   
15:  $\Delta_t = 0$   
16: end if  
17: end for
Again,  $\text{peek}(\mathbf{n}(t_s))$  retrieves the highest-priority element without extracting it from the heap, which in this case is the minimum over all neurons' next unperturbed spike time. An example implementation of a heterogeneous QIF network with SparseProp in this time-based representation is available here.
Note that this can also easily be extended to mixed networks of excitatory and inhibitory neurons (or to  $k$  population networks with different external input [42]).
The update of the postsynaptic neurons in line 11 is analogous to the evaluation of the phase transition curve in the homogeneous SparseProp, but takes into account the different phase velocities. For example, in the case of a QIF neuron, it takes the form
$$
\operatorname {u p d a t e} \left(n _ {i ^ {*}} ^ {-} \left(t _ {s}\right), \Delta_ {t}\right) = \frac {\pi - g \left(\pi - \left(n _ {i} - \Delta_ {t}\right) \omega_ {i}\right)}{\omega_ {i}} + \Delta_ {t} \tag {11}
$$
Here, the phase transition curve, denoted as  $g(\phi_i)$ , is the same as previously described in Eq. 10:  $g(\phi_i) = 2\arctan \left(\tan \frac{\phi_i}{2} + c\right)$  with effective coupling strength  $c$ . For the QIF neuron, the period  $T_i$  is given by  $T_i = 2\pi / \omega_i$ , with  $\omega = 2\sqrt{I^{\mathrm{ext}}}$ .
7 Efficient Event-Based Training of Spiking Neural Networks
SparseProp can also be used for efficient event-based training of spiking neural networks. While surrogate gradients seem to facilitate training spiking networks [43, 44, 45, 46], it was recently claimed that EventProp, which uses exact analytical gradients based on spike times, can also yield competitive performance [47]. To obtain precise gradients for event-based simulations, this algorithm makes use of the adjoint method from optimization theory [48, 49]. In this approach, in the forward pass the membrane potential dynamics (Eq. 1) is solved in an event-based simulation and spike times are stored. In the backward pass, an adjoint system of variables is integrated backward in time, and errors in spike times are back-propagated through the size of the jumps in these adjoint variables at the stored spike times. For event-based simulations of neuron models where the next spike time is analytically known or can be obtained like in section 5, the entries adjoint integration is being taken care of by differential programming [47]. Besides the previously mentioned advantages of event-based simulation regarding precision, this approach is not only sparse in time but also sparse with respect to the synapses, as only synaptic events are used.
We suggest here to further improve the efficiency of event-based training using SparseProp. As the backward pass has the identical amortized time complexity of  $\mathcal{O}(\log (N))$  per network spike as the forward pass, we expect a significant boost of training speed for large sparse networks [47, 50, 51]. Regrettably, the implementation of EventProp presented in [47] was not publicly available during the time of submission. In contrast, the authors of [50] made their re-implementation of EventProp publicly available, but they did not use an event-based implementation but forward Euler to integrate the network dynamics. It will be an important next step to also benchmark the performance of SparseProp training. As the gradients are identical, we expect similar results to [47, 50, 51]
In this work, we focus exclusively on recurrent networks of pulse-coupled univariate neuron models such as the quadratic integrate-and-fire neuron, leaky integrate-and-fire neuron, and the exponential integrate-and-fire neuron thereby excluding the exploration of multivariate neuron models. Extending our approach to encompass multivariate models, such as the leaky- or quadratic integrate-and-fire neurons with slow synapses [19, 28, 51, 52], remains a promising direction for future research.
A more fundamental limitation of our study is the apparent incompatibility of event-based simulations with surrogate gradient techniques [46, 53]. While one could introduce 'ghost spikes' in the event-based simulation to emulate surrogate gradients when neurons hit a lower threshold. However, it remains unclear how to preserve the favorable computational scaling of SparseProp in this case.
Furthermore, SparseProp requires an analytical solution of  $V_{i}(t)$  given the initial state and the neuronal dynamics  $\dot{V}_{i}(t)$  is necessary. This limitation excludes conductance-based models [54, 55, 56], for example. Additionally, time-varying input can only be introduced in the form of deterministic currents or point processes, such as Poisson input. Incorporating stochastic input without an analytical solution of  $V_{i}(t)$  would require further development.
Moreover, the reduction in computational cost per network spike from  $\mathcal{O}(N)$  to  $\mathcal{O}(\log (N))$  can only be achieved in sparse networks, where the number of synapses per neuron  $K$  is much smaller than the
total number of neurons  $N$ . For dense networks where the number of synapses scales proportionally to the number of neurons [13], a priority queue implemented by a binary heap is disadvantageous compared to a conventional array in the large network limit, as every network spike involves changing the priority of  $\mathcal{O}(N)$  neurons thus  $\mathcal{O}((N + 1)\log (N))$  flops per network spike which corresponds to  $\mathcal{O}((N + 1)\cdot N\log (N))$  flops for a fixed simulation time. A batch-update of all postsynaptic neurons might be faster for very large networks [57] but this is beyond the scope of this work. It would involve  $\mathcal{O}\left(K + \log (K)\log (N)\right)$  flops corresponding to  $\mathcal{O}\left(N + \log (N)\log (N)\right)$  in dense networks. In the case of purely excitatory sparse spiking networks, a Fibonacci heap might be a more efficient implementation in the large network limit [58], as the 'decrease key' operation takes constant time  $\mathcal{O}(1)$  compared to  $\mathcal{O}(\log (N))$  in the case of a binary heap. Note that for practical purposes, the asymptotic scaling of the computational complexity of Fibonacci heaps has an unfavorably large prefactor [59]. Therefore, other heap structure implementations might be faster [60]. While large mammalian brains are sparse  $N\gg K$ , for models of small brains or local circuits with dense connectivity the conventional event-based simulations might be faster.
Additionally, SparseProp cannot currently model biophysically detailed neuron features, including dendrites, threshold adaptation, slow synaptic timescales, and short-term plasticity. Incorporating these features would be another valuable avenue for future computational neuroscience research.
Lastly, we did not yet perform a comprehensive systematic benchmark of training with challenging tasks. Such benchmarks would offer further insights into the training performance of our proposed framework.
We introduce SparseProp, an efficient and numerically exact algorithm for simulating and training large sparse spiking neural networks in event-based simulations. By exploiting network sparsity, we achieve a significant reduction in computational cost per network spike from  $\mathcal{O}(N)$  to  $\mathcal{O}(\log(N))$ . This speedup is achieved by optimizing two critical steps in the event-based simulation: finding the next spiking neuron and evolving the network state to the next network spike time. First, we employ a binary heap data structure for finding the next spiking neuron and updating the membrane potential of postsynaptic neurons. Second, we utilize a change of variables into a co-moving reference frame, avoiding the need to iterate through all neurons at every spike.
Our results demonstrate the scalability and utility of SparseProp in numerical experiments. We demonstrate that SparseProp speeds up the simulation and training of a sparse SNN with one million neurons by over four orders of magnitude compared to previous implementations. In contrast to the conventional event-based algorithm, which would require three CPU years to simulate 100 seconds of network dynamics, SparseProp achieves the same simulation in just one CPU hour.
This advancement enables the simulation and training of more biologically plausible, large-scale neural network models relevant to theoretical neuroscience. Furthermore, it might pave the way for the exploration of large-scale event-based spiking neural networks in other machine learning domains, such as natural language processing [61], embedded automotive applications [62], robotics [63], and in silico pretraining of neuromorphic hardware [1, 64].
The impact of SparseProp extends to the recent surge of training for spiking neural networks in theoretical neuroscience [50, 65], machine learning [61, 66, 67], and neuromorphic computing [68, 69, 70]. This algorithm is expected to substantially accelerate the training speed in recent works focused on event-based spiking networks [47, 50, 51].
Future extensions of SparseProp to multilayer and feedforward networks would be a promising avenue. Integration of SparseProp with recent attempts in biologically more plausible backpropagation and real-time recurrent learning [53, 65, 71, 72, 73, 74] could be used in future research in theoretical neuroscience. To improve training performance, future studies should consider incorporating additional slow time scales, such as slowly decaying synaptic currents [72], and slow threshold adaptation [65], which appear to enhance temporal-credit assignment in recurrent neural networks for rudimentary tasks [53, 65, 72]. Additionally, incorporating gradient shaping and insights from advances in event-based spiking network training [50, 51] could yield further gains in performance.
I thank S. Goedeke, J. Liedtke, R.-M. Memmesheimer, M. Monteforte, A. Palmigiano, M. Puelma Touzel, A. Schmidt, M. Schottdorf, and F. Wolf for fruitful discussions. Moreover, I thank L.F. Abbott, S. Migirditch, A. Palmigiano, Y. Park, and D. Sussillo for comments on the manuscript. Finally, I thank the Julia community.
Research supported by NSF NeuroNex Award (DBI-1707398), the Gatsby Charitable Foundation (GAT3708), the Simons Collaboration for the Global Brain (542939SPI), and the Swartz Foundation (2021-6). This work was further supported by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) 436260547 in relation to NeuroNex (National Science Foundation 2015276) and under Germany's Excellence Strategy - EXC 2067/1-390729940, DFG - Project-ID 317475864 - SFB 1286, by Evangelisches Studienwerk Villigst, by the Leibniz Association (project K265/2019), and by the Niedersächsisches Vorab of the VolkswagenStiftung through the Göttingen Campus Institute for Dynamics of Biological Networks.
[1] Peter U. Diehl, Guido Zarrella, Andrew Cassidy, Bruno U. Pedroni, and Emre Neftci. Conversion of artificial recurrent neural networks to spiking neural networks for low-power neuromorphic hardware. In 2016 IEEE International Conference on Rebooting Computing (ICRC), pages 1-8, October 2016.
[2] Payal Dhar. The carbon impact of artificial intelligence. Nature Machine Intelligence, 2(8):423-425, August 2020.
[3] Tim Golisch and Markus Meister. Rapid Neural Coding in the Retina with Relative Spike Latencies. Science, 319(5866):1108-1111, February 2008.
[4] Robert Gutig and Haim Sompolinsky. The tempotron: a neuron that learns spike timing-based decisions. Nature Neuroscience, 9(3):420-428, March 2006.
[5] Robert Gütig. Spiking neurons can discover predictive features by aggregate-label learning. Science, 351(6277):aab4113, March 2016.
[6] D. Hansel, G. Mato, C. Meunier, and L. Neltner. On Numerical Simulations of Integrate-and-Fire Neural Networks. Neural Computation, 10(2):467-483, February 1998.
[7] D. J. Amit, M. R. Evans, and M. Abeles. Attractor neural networks with biological probe records. Network: Computation in Neural Systems, 1(4):381, October 1990.
[8] M V Tsodyks and T Sejnowski. Rapid state switching in balanced cortical network models. Network: Computation in Neural Systems, 6(2):111-124, January 1995.
[9] Nicolas Brunel. Dynamics of Sparsely Connected Networks of Excitatory and Inhibitory Spiking Neurons. Journal of Computational Neuroscience, 8(3):183-208, May 2000.
[10] Michael Monteforte and Fred Wolf. Dynamical Entropy Production in Spiking Neuron Networks in the Balanced State. Physical Review Letters, 105(26):268104, December 2010.
[11] Michael Monteforte and Fred Wolf. Dynamic Flux Tubes Form Reservoirs of Stability in Neuronal Circuits. Physical Review X, 2(4):041007, November 2012.
[12] Wulfram Gerstner, Werner M. Kistler, Richard Naud, and Liam Paninski. *Neuronal Dynamics: From Single Neurons to Networks and Models of Cognition*. Cambridge University Press, Cambridge, uk ed. edition edition, September 2014.
[13] Alfonso Renart, Jaime de la Rocha, Peter Bartho, Liad Hollender, Nestor Parga, Alex Reyes, and Kenneth D. Harris. The Asynchronous State in Cortical Circuits. Science, 327(5965):587-590, January 2010.
[14] Agostina Palmigiano, Rainer Engelken, and Fred Wolf. Boosting of neural circuit chaos at the onset of collective oscillations. eLife, 12, November 2023.
[72] James M Murray. Local online learning in recurrent networks with random feedback. eLife, 8:e43299, May 2019.
[73] Alexandre Payeur, Jordan Guerguiev, Friedemann Zenke, Blake A. Richards, and Richard Naud. Burst-dependent synaptic plasticity can coordinate learning in hierarchical circuits. Nature Neuroscience, 24(7):1010-1019, July 2021.
[74] Yuhan Helena Liu, Stephen Smith, Stefan Mihalas, Eric Todd SheaBrown, and Uygar Sumbul. Biologically-plausible backpropagation through arbitrary timespans via local neuromodulators. October 2022.
[75] Paul Manz, Sven Goedeke, and Raoul-Martin Memmesheimer. Dynamics and computation in mixed networks containing neurons that accelerate towards spiking. *Physical Review. E*, 100(4-1):042404, October 2019.
[76] Eugene M. Izhikevich and Gerald M. Edelman. Large-scale model of mammalian thalamocortical systems. Proceedings of the National Academy of Sciences, 105(9):3593-3598, 2008.
[77] Robert Rosenbaum and Brent Doiron. Balanced Networks of Spiking Neurons with Spatially Dependent Recurrent Connections. Physical Review X, 4(2):021039, May 2014.
[78] William R. Softky and Christof Koch. Cortical Cells Should Fire Regularly, But Do Not. Neural Computation, 4(5):643-646, September 1992.
[79] W. R. Softky and C. Koch. The highly irregular firing of cortical cells is inconsistent with temporal integration of random EPSPs. The Journal of Neuroscience, 13(1):334-350, January 1993.
[80] W. H. Calvin and C. F. Stevens. Synaptic noise and other sources of randomness in motoneuron interspike intervals. Journal of Neurophysiology, 31(4):574-587, July 1968.
[81] H L Bryant and J P Segundo. Spike initiation by transmembrane current: a white-noise analysis. The Journal of Physiology, 260(2):279-314, September 1976.
[82] John D. Hunter, John G. Milton, Peter J. Thomas, and Jack D. Cowan. Resonance Effect for Neural Spike Time Reliability. Journal of Neurophysiology, 80(3):1427-1438, September 1998.
[83] Z. F. Mainen and T. J. Sejnowski. Reliability of spike timing in neocortical neurons. Science, 268(5216):1503-1506, June 1995.
[84] Michael N. Shadlen and William T. Newsome. Noise, neural codes and cortical organization. Current Opinion in Neurobiology, 4(4):569-579, August 1994.
[85] Michael N. Shadlen and William T. Newsome. The Variable Discharge of Cortical Neurons: Implications for Connectivity, Computation, and Information Coding. The Journal of Neuroscience, 18(10):3870-3896, May 1998.
[86] C. van Vreeswijk and H. Sompolinsky. Chaos in Neuronal Networks with Balanced Excitatory and Inhibitory Activity. Science, 274(5293):1724-1726, December 1996.
[87] C. van Vreeswijk and H. Sompolinsky. Chaotic Balanced State in a Model of Cortical Circuits. Neural Computation, 10(6):1321-1371, 1998.
[88] Tom Tetzlaff, Moritz Helias, Gaute T. Einevoll, and Markus Diesmann. Decorrelation of Neural-Network Activity by Inhibitory Feedback. PLOS Comput Biol, 8(8):e1002596, August 2012.
A SparseProp Code for Efficient Spiking Network Simulation in Julia
This minimal code implements the SparseProp algorithm for simulating a spiking network of  $N = 10^{5}$  LIF neurons.
using DataStructures, RandomNumbers. Xorshifts, StatsBase, PyPlot
function lifnet(n,nstep,k,j0,ratewnt, $\tau$ ,seodic,seednet)  
    iext =  $\tau$ *sqrt(k)*j0*ratewnt/1000 # iext given by balance equation  
     $\omega, c = 1/log(1. + 1/iext)$ ,j0/sqrt(k)/(1. + iext) # phase velocity LIF  
     $\phi$ th,  $\phi$ shift = 1., 0. # threshold for LIF  
    r = Xoroshiro128Plus(seodic) # init. random number generator  
# initialize binary heap:  
     $\phi =$  AdjustableBinaryHeap{Float64, DataStructures.FasterReverse} (rand(r,n))  
    spikeidx = Int64[] # initialize time  
    spiketimes = Float64[] # spike raster  
    postidx = rand(Int,k)  
for s = 1 : nstep # main loop  
     $\phi$ max, j = top_with_handle( $\phi$ ) # get phase of next spiking neuron  
    d $\phi =$ $\phi$ th -  $\phi$ max -  $\phi$ shift # calculate next spike time  
     $\phi$ shift += d $\phi$  # global shift to evolve network state  
    Random.seed!(r,j+seednet) # spiking neuron index is seed ofrng  
    sample!(r,1:n-1, postidx; replace=false) # get receiving neuron index  
@inbounds for i = 1:k # avoid autapses  
    postidx[i] >= j && (postidx[i] += 1)  
end  
ptc!( $\phi$ , postidx,  $\phi$ shift,  $\omega, c$ ) # evaluate phase transition curve  
update!( $\phi, j, -\phi$ shift) # reset spiking neuron  
push!(spiketimes,  $\phi$ shift) # store spike times  
push!(spikeidx, j) # store spiking neuron index  
end  
nstep/ $\phi$ shift/n/ $\tau$ * $\omega$ , spikeidx, spiketimes* $\tau/\omega$  # output: rate, spike times & indices
function ptc!( $\phi$ , postid,  $\phi$ shift,  $\omega$ , c) # phase transition curve of LIF for i = postid  $\phi[i] = -\omega * \log(\exp(-(\phi[i] + \phi shift)/\omega) + c) - \phi shift$  (#(Eq. 12) end end
set parameters:  
#n: # of neurons, k: synapses/neuron, j0: syn. strength, τ: membr. time const.  
n,nstep,k,j0,ratewnt,τ,seodic,seednet = 10^5,10^5,100,1,1.,.01,1,1  
# quick run to compile code  
@time lifnet(100, 1, 10, j0, ratewnt, τ, seedic, seednet);
run & benchmark network with specified parameters
GC.gc(); @time rate, sidx, stimes = lifnet(n, nstep, k, j0, ratewnt,  $\tau$ , seedic, seednet)
plot spike raster plot(stimes,sidx,",k",ms=0.1) ylabel("Neuron Index",fontsize=20) xlabel("Time(s)",fontsize=20);tight.layout()
B Event-Based SparseProp With Numerical Phase Transition Curve
For neuron models where the next spike time cannot be determined analytically and/or the network state cannot be temporally evolved based on a closed-form expression  $V(t_{s + t})$ , such as the exponential integrate-and-fire model, we present an event-based implementation that has the same computational cost of  $\mathcal{O}\left(K\log (N)\right)$  per network spike. To achieve this, we evaluate the phase transition curve either by employing a lookup table or by approximating it using Chebyshev polynomials. We initially describe the implementation based on a lookup table and provide an example implementation:
We begin by numerically integrating the ordinary differential equation that describes the membrane potential  $V$  of the neuron model, without considering any external input spikes:
$$
\tau_ {\mathrm {m}} \frac {\mathrm {d} V}{\mathrm {d} t} = F (V) + I ^ {\text {e x t}}. \tag {12}
$$
The integration is performed from the reset voltage  $V_{\mathrm{re}}$  to the threshold voltage  $V_{\mathrm{th}}$  as follows:
$$
V (t) = \int_ {0} ^ {t} (F (V) + I ^ {\mathrm {e x t}}) / \tau_ {\mathrm {m}}. \tag {13}
$$
We obtain the numerical solution  $V(t)$  by employing the DifferentialEquation.j1 package with a callback that terminates the integration precisely at  $V_{\mathrm{th}}$ .