Online Learning Of Neural Computations From Sparse Temporal Feedback
Authors: Lukas Braun
Tim P. Vogels
Venue: NeurIPS 2021 Poster (2021)
Keywords: spiking neural networks
event-dependent scaling
intrinsic parameters
online learning
teacher-student paradigm
surrogate gradient

Abstract: Neuronal computations depend on synaptic connectivity and intrinsic electrophysiological properties. Synaptic connectivity determines which inputs from presynaptic neurons are integrated, while cellular properties determine how inputs are filtered over time. Unlike their biological counterparts, most computational approaches to learning in simulated neural networks are limited to changes in synaptic connectivity. However, if intrinsic parameters change, neural computations are altered drastically. Here, we include the parameters that determine the intrinsic properties, e.g., time constants and reset potential, into the learning paradigm. Using sparse feedback signals that indicate target spike times, and gradient-based parameter updates, we show that the intrinsic parameters can be learned along with the synaptic weights to produce specific input-output functions. Specifically, we use  a teacher-student paradigm in which a randomly initialised leaky integrate-and-fire or resonate-and-fire neuron must recover the parameters of a teacher neuron. We show that complex temporal functions can be learned online and without backpropagation through time, relying on event-based updates only. Our results are a step towards online learning of neural computations from ungraded and unsigned sparse feedback signals with a biologically inspired learning mechanism.

Online Learning Of Neural Computations From Sparse Temporal Feedback
Lukas Braun 1.2 lukas braun@psy. ox. ac. uk
Tim P. Vogels
tim. vogels@ist. ac. at
Abstract
Neuronal computations depend on synaptic connectivity and intrinsic electrophysio- logical properties. Synaptic connectivity determines which inputs from presynaptic neurons are integrated, while cellular properties determine how inputs are filtered over time. Unlike their biological counterparts, most computational approaches learning in simulated neural networks are limited to changes in synaptic connec- tivity. However, if intrinsic parameters change, neural computations are altered drastically. Here, we include the parameters that determine the intrinsic properties, e.g., time constants and reset potential, into the learning paradigm. Using sparse feedback signals that indicate target spike times, and gradient-based parameter updates, we show that the intrinsic parameters can learned along with the synaptic weights to produce specific input-output functions. Specifically, we use a teacher-student paradigm in which randomly initialised leaky integrate-and fire resonate-ande neuron must recover the parameters of a teacher neuron. We show that complex temporal functions can be learned online and without backprop- agation through time, relying event-based updates only. Our results a step towards online learning of neural computations from ungraded and unsigned sparse feedback signals with biologically inspired learning mechanism.
Introduction
In biological and artificial spiking neurons, information processing depends on synaptic connectivity as well as intrinsic variables. How a neuron integrates synaptic inputs over time changes its input- output function, leading to large variety of neural computations [T]. In biology. the input-output function of a neuron modified not only by synaptic plasticity (2), but also by alterations of the neuron's electrophysiology, e.g., through changes in the composition of rransmembrrnne ion channels, so-called intrinsic plasticity 3 4 Synaptic interactions influence the amount of neurotransmitter released from the presynaptic neuron the amount of receptors in the postsynaptic terminal and therefore how strongly neurons are connected to each other. Intrinsic properties of the neuron, on the other hand, affect the neuron's threshold and resting potential and how strongly postsynaptic potentials are amplified 4
Correspondingly, the computational properties of an artificial neuron depend on the values of its synaptic weights and intrinsic parameters. For instance, the neural computation performed by a leaky integrate-an and--ndd-i fire neuron [5] a leaky resonate-and and-fire neuron 6 with identical synaptic inputs and synaptic weights but different intrinsic parameters varies strongly within and across neuron models (Figure A). Learning in artificial neurons is thus the process of modifying set of synaptic weights and intrinsic parameters to reach target input-output function. However, unlike their biological counterparts, most computational approaches learning in spiking neurons limited effects of
1. Department of Experimental Psychology. University of Oxford, Oxford, United Kingdom
2. Bernstein Center for Computational Neuroscience Berlin, Berlin, Germany Institute of Science and Technology Austria, Klosterneuburg, Austria
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

A
B Inputs
Integrate-and
Outputs
Threshold
Te ms
VIUI
Teacher
Inputs
55
Vase -02
Threshold
Resonate-ano-ff neuron
-b 1/ms - 10 Hz
Student
Hit
Miss
Wi
Viewe
Inputs
-b- 22 1/ms
Error Signal
d'
200
400
600
800 1000 Time [ms]
1200
1400
Frese =
Prost -1
Figure 1: Neural computations and the teacher-student paradigm. A: Despite identical synaptic weights and synaptic inputs, the input-ouiput function varies strongly within and across neuron models due to different intrinsic parameters. B: The teacher-student paradigm is used to study the convergence properties of a learning algorithm. The teacher and the student receive identical inputs, but their synaptic weights and intrinsic parameters have different values and therefore generate different outputs. The learning problem for the student neuron is to recover the fixed set parameters of the teacher and thus to learn the neural computation performed by the teacher neuron.
alterations in synaptic connectivity. Here, we derive gradient-based online learning rule for leaky integrate-ande and-fire neurons and leaky resonate-and-Fi neurons that can learn the synaptie connectivity along their intrinsic parameters from sparse temporal feedback in the teacher-student paradigm.
1.1 Related Work
Applying gradient-based optimisation and the backpropagation algorithm 7 spiking neurons is difficult since the gradient of spike trains is ill-defined: it is zero everywhere except at spike times, where it is infinite. Methods to overcome the problem of non-differentiable spike trains include using gradients with respect to the probability density of spike-times
[9], minimising the temporal distance between output and target spike times [IO]. temporal smoothing of synaptic-currents [ or the discrete eтror signals caused by target-spikes [2 14 or by temporal gradient accumulation using the adjoint method [13]. In contrast to approaches that are directly optimising output spike times, there are also approaches that optimise the activity of spiking neurons through their network activity [16] 177, firing rate response function [18] or via the auxiliary objective of generating fixed amount of output spikes for certain input spike patterns [19 (2). Alternatively, so-called "surrogate gradient" methods replace the non-differentiable threshold function with a smooth surrogate function during the backpropagation of errors [21 22 23 Using surrogate gradients and backpropagation
through time 24), multilayer spiking neural networks can successfully trained on wide range of even when errors are backpropagated at spike times only (28)
supervised learning tasks 25 26 27
Our learning rule is closely related to the work by Gütig and Sompolinsky [19] and Memmesheimer et al. [29]. The tempotron learning rule [19] trains leaky integrate-and-fing neurons to emit an output spike for some input spike patterns, but remain silent for others. Memmesheimer et al. 29 adapted this learning rule to learn synaptic weights that map input spike trains to a target output spike train. In contrast to the tempotron learning rule, the feedback signal does not indicate the class membership of an input spike pattern but target spike times: on the occurrence of each missed or false output spike the distance to the spiking threshold minimised or respectively, maximised. If an output spike does not fall within a tolerance window around target spike, a single parameter update is performed and the training procedure and neuron model are reset to the beginning of the training sequence. Consequently, the target spike train is learned one by one, from the first to the last spike of the target sequence, eventually requiring many replays of the sequence until the algorithm converges.
We extend the learning rule by Memmesheimer et al. 29 by an event-dependen scaling factor, such that synaptic weights and intrinsic parameters can be learned mutually and online, i.e., from a continuous stream of inputs without resetting the training procedure after each error and without relying on a tolerance window. Like Memmesheimer al. (29), we train without backpropagating errors through time (24). We hypothesise that learning signal that indicates when neuron should spike, paired with a gradient-based online parameter update rule, is sufficient to learn the complex temporal onput-output relationships of a neuronal computation.

---

2 Methods
2.1 Teacher-student paradigm
Learning in artificial neurons relies on finding the set of parameters that facilitates certain target computation. One approach to test an algorithm's 's learning capability to test its ability to map randomly sampled input spike patterns
randomly sampled output spike trains (e.g. used in [2] 23] and in [30 to compare algorithms). However, under this testing paradigm, the difficulty of the task is directly dependent on the statisties (e.g. dimensionality and firing rate) of the input spike patterns and target spikes. Therefore, the size and shape of the solution space and thus the difficulty of the task unknown and strongly fluctuate with the choice of hyperparameters. As consequence, the set of parameters that solves the problem may be large and thus even suboptimal learning algorithms can still yield good performance. For example, some of the solutions learning task may not be anticipated, like networks with biologically implausible high firing rates (27]. Further, using random input-output spike patterns of fixed length unsuitable for evaluating an online-learning algorithm since, as the length of the random pattern increases, the probability of solution existing vanishing to zero.
The teacher-student paradigm 31 32 circumvents this problem by generating target spike times using a model (i.e. the teacher neuron). The learning task is then to recover the teacher's parameter that determine its neural computation with student neuron (e.g. in (29) "Reconstruction of Synaptic Weights").. Like this, the learning problem well defined, single point in parameter space. In case, learning signals are derived from comparing the output spike train of the teacher neuron with the output spike train of the student (Figure [B). The teacher-student paradigm has several advantages, e.g., over the test performance in a standardised supervised learning task. First, if the teacher and student neuron have the same architecture, it is guaranteed that unique solution to the training problem exists. Second, the student's 's parameters, and thus performance, can be compared to the solution during and after the training process. Finally, since the teacher used generate training samples, an infinite amount of unique and continuous training data can be sampled. In summary, by controlling the solution space of the task, the teacher-student paradigm can be used to precisely test the capability of performing spike-patter recognition and the validity of assumptions of a learning algorithm.
D
14-
6:10
To
13-
20ma
40ms
2-tr
ditr
s0ms
Bitr
a.s
2.0
Patonnaa
55
0:00
03 00 03 02
A."
se
Pestoyappte Terenial
630-
-b
Str
6.5
20me
10ms
Lirra
Linrs
2.1
2.4
8.2
1.3
602-
2.1
-0.5
3.3
3.0
03
(ms]
Resorant Current
Time [ms]
Figure 2: Methods, A: Event-dependent scaling factor. B: The SuperSpike 23 surrogate gradient for different 3 values. C: Influence of the membrane (Tm) and synaptic time constant (r₂) on the membrane potential. While the former determines how quickly the membrane potential falls back to rest, the later shifts the position of the peak amplitude. D: Phase plane of the leaky resonate-and-finn neuron with example trajectory (green), separatrix (orange) and post spike reset position. E: Influence of the frequency of the subthreshold potential w and damping factor on the membrane potential, F: normnlr distributions with 3+ 1.2 and examplary sample of 80 excitatory (red) and 20 inhibitory (blue) weights. G: A weight with value 0.1 (LIF, blue) and -0.1 (LRF, green) was sampled from the log-normal distribution. After rescaling the weight with the normalisation factor x the peak post synaptic potential is equal to the sampled value, independent of intrinsic parameter values.
3

---

2.2 Error signals
As the dynamics of the teacher and student neuron evolve over time, three possible events can oceur when comparing their output spike trains (Figure [B). First, the student may generate a spike at the same time the teacher (a hit). Second, the teacher may generate spike when the student does not (a miss, denoted by d= = -1). Third, the student may generate a spike when the teacher does not (a false positive, denoted by d+ =)) Since false positives can be detected by the student neuron without feedback, only missed spikes require error signals between teacher and student neurons that are ungraded, unsigned and therefore "unary". Intuitively, a miss occurs when the student's membrane potential V is too low and a false positive occurs when the student's membrane potential too high. We base learning on this observation and update the student's parameters such that the distance to the threshold is reduced for each míss, but increased for each false positive spike [1]] 29
2.3 The event-dependent scaling online learning rule
Our learning rule consists of gradient-based parameter update like in Memmesheimer et al. 22 (for derivation see appendix A) and an additional event-dependent scaling factor X x(d+). We thus call it the Event-Dependent Scaling (EDS) rule. More specifically, we use the partial derivative of the membrane potential with respect to a parameter and perform a gradient ascent step on each miss (d" 1) and gradient descent step on each false positive (d+ 1) spike:
A -X(Ad )a (t)
ae
Here, the EDS factor X(At) scales parameter updates according to the truncated sigmoidal function
mmin(Ad Ao)
x(A+) = 1000 1000 exp
(n(().)
m
where As cutoff parameter, m determines the midpoint and b the slope of the sigmoidal and Ad= denotes the time since the last parameter update. Thus, the size of the parameter update is event-dependent, In practice, In contrast to the EDS rule, surrogate gradient methods [23] [6] (2) use a scaling function X(V(t)) that based on the current membrane potential. We compare the EDS factor A(Ad=) to the the SuperSpike surrogate gradient X(V) (8|V - VThll + 1) (23].
Ag = 75, 500 and b = 4, which is plotted in figure A
Using the EDS rule, parameter updates temporal, stochastic and online. Updates are temporal since the underlying function that we optimise is a function evolving over time i.e. the parameterised function that deseribes the membrane potential. They stochastic since updates are performed for a single teacher-student pair and not as averages over multiple initialisations. And lastly, they are online since gradients are calculated for the current point in time only, without propagating errors through time [24) and therefore without taking the recurrent nature of neural dynamics into account (cf. (26)) ) To speed up learning, we use the Adam optimiser (33) with default parameters (3₁ 0.9, 32 = 0.999, e = le *) and per parameter learning rates, which are scaled according to the range from which target values are sampled (Nw 35e 17,, 7e -4, Th 28e TN; = 7 and
Ma Se 1/b = 15e The = 33e nV, = 8e -5 nj, 8e -5-
2.4 Neuron models
Throughout the experiments, we use two neuron models: the leaky integrate-and-fire (LIF) neuron [5] and the leaky resonate-and-Fe (LRF) neuron [6] with current-based synaptic inputs. Both of the models are based on linear subthreshold dynamies, fixed spiking threshold and post spike reset rule: Whenever the linear dynamics reach the threshold value, the neuron elicits an output spike and its dynamics are reset according to the reset rule. For simplicity, the membrane potential V of both models is rescaled such that the distance from the resting potential (V, = 0) and the spiking threshold Vinr is 1. Input, output and target spikes are modelled as Dirac delta spike trains S()) Σ(e) 8( 8(Ai). where At = t f substitutes the time that has elapsed since spike event f, and (t} is a shorthand notation for the set of n spike times of a spike train (ts k = 1, 2, ..., n and t, I t}. The condition 4 l preserves temporal causality.
Solving the linear system of differential equations that determines the subthreshold voltage dynamics of the leaky integrate-and-fie (LIF) neuron and adding the spike reset rule (appendix B.2 yields

---

the selke-response model [3]. It describes how the membrane potential V of neuron j evolves over time:
N
A/
Vy(t) = Σ
wij Σ
K(AC) + (V.
V(te)
Σ
exp
(3)
Tm
(4.
(i))
with double exponential voltage kernel
At Tm
At
K(At) exp
exp
(4)
Ts
input spike times E, output spike times i, and spiking threshold Vin. The synaptic weights Way deter- mine how strongly the membrane potential V is perturbed by synaptic inputs and if the perturbation excitatory or inhibitory. The synaptic time constant Ta and membrane time constant Tm determine how quickly synaptic currents and the membrane potential return to rest (Fig. (C) Together with the reset potential Vr, they determine the intrinsic dynamies of the neuron. During training, the intrinsic parameters (Ts, Tm, V/) are optimised along with synaptic weights.
The temporal dynamics of the leaky resonate-and-five (LRF) neuron 66 provide model for phenom- ena observed in biological neurons that can not be generated by LIF neuron []]. most prominently tob-threshold oscillations of the membrane potential 35]. This makes LRFs sensitive to inputs of certain frequency, whereas LIF neurons are sensitive to the rate of inputs. Solving the system of the two coupled linear differential equations (appendix (B.)) that describe the oscillating subthreshold potential V and the resonant current yields
V5(1)
Σ Wi
exp(Ai,b) sindf,b)sinnn (iiw) + (x + sin(Aijw)); (5) (1,1))
Σ
with input spike times f and most recent output spike time i, (Fig. 2D). Akin to a pendulum, the temporal dynamics of the LRF neuron perturbed by incoming spikes and scaled by the synaptic weights Wyy- The frequency of the subthreshold oscillations is determined by w 2 0, and the dynamics return rest based on the damping factor b (Fig. 3). In contrast the LIF neuron, the synaptic currents reset after each spike, i.e., the dynamies set I, V, on the phase plane. Consequentially, in equation 5 we sum only over input spikes since the most recent output spike {4 t.}. In summary, the learnable parameters of the LRF neuron are the synaptic weights Wyj. the damping factor b, the frequency of the subthreshold oscillations w and the reset values I, and V,.
2.5 Parameter initialisations
Input spike trains are homogeneously Poisson distributed. In total, there are 100 synaptic inputs and the ratio between excitatory and inhibitory inputs is 4: 1 [36]. Excitatory and inhibitory input firing rates differ accordingly, i.e., n/ = 10Hz and rm 40Hz.
The mean Ph and variance o² of the distribution are set so that approximately 99% of the cumulative distribution function 0.2 and thus below one-fifth of the distance between resting potential and threshold (appendix C Further, use the bias term 3+ to fit the target output firing rate Tout by creating small imbalance between excitatory and inhibitory weights (Fig. 2):
Synaptic weights are sampled independently from a log-normal distribution
37
Wexe +* (ognormal(uw,,0] and Winh Lognormal 2.)
(6)
If weight > 0.3, it is resampled. Weights are then scaled once with the normalisation factor k 38 p. 143], in order for the peak amplitude of the postsynaptic potential to be equivalent to the size of the sampled weight, independent the values the intrinsic parameters (Fig. 2G). Note, that x is different for leaky integrate-and-finn and leaky resonate-and and-tee--d-frre neurons (appendix D) To ensure balance between excitation and inhibition, target values are resampled as described in section 2.5 until there is 3+ 2.5 that reaches the target output firing rate Fout- Importantly, the position of excitatory and inhibitory weights is randomised, i.e. weights do not adhere to Dale's principle (39 during learning and flip sign.
The teacher neuron parameters determine the target computation and are not altered during the training process. Both the student's 's initial values and the teacher's parameter values are chosen
5

---

in the following way: For Integrate-and-fire neurons, the membrane time constant is sampled from Tm (10ms, 60ms). The synaptic time constant т, smaller than the membrane time constant and set to (1--1.5. 0.9) and Tout ((Hz, 50HZ) respectively. For Resonate-and-fire neurons, the damping factor -b ~ U(20) fims, 120 /ms) and the frequency of the subthreshold oscillations w ~ U(2Hz 25Hz) repeatedly sampled until 4. The two reset parameters I, and V, are both sampled from 4(-0.8, 0.8). Target output firing rates are sampled from Yout ~ (1Hz, 20Hz).
Tm/ /4. Reset potentials and target firing rates are sampled from V, ~
2.6 Convergence and robustness
A simulation's 's convergence is determined by the signed relative error between the student's 0 and teacher's 0' parameter values:
(o, 0')
(0, max (1 12 0.075)
where 0.075 in the denominator is included to prevent division by The convergence time the moment the absolute value of the signed relative error |e] falls below and stays below the parameter dependent threshold values Og for all parameters. For LIF neurons the target threshold values are _. = yv, = 0.15 and θ, = O = 0.025 and for LRF neurons O₂ = 0.05, O₂ = O = 0.025 and θv, = I, 0.1.
The robustness of the learning rule tested by adding temporal noise & to the spike time of the teacher signal. The noise is sampled from Gaussian distribution & [N( 0, of) where L-T denotes rounding to the nearest integer and dg is the standard deviation of the noise in ms.
2.7 Implementation, simulation and code availability
All simulations are run for = different random seeds discretised time at a temporal resolution of Ims. Reported times refer to simulated and not real time, i.e. the time required to run simulation was several orders of magnitude faster. For example,
independent simulations of 12,000 minutes of simulated time at Ims temporal resolution (as in Fig. 33. take less than 30 minutes of real time on a single AMD Ryzen 5950x. C++ code to replicate all simulations and plots is publicly available' under GPLv3 license and uses the MPL 2.0 licensed Eigen software library v3.3.7 [40].
3 Results
The results are separated into a theoretical and an empirical part. In the theoretical part, we derive the partial derivatives (sec. 3.11) that are required for the parameter update rule (eq. I and motivate their event-based versions (sec. 3.2 and derivation in appendix E). In the empirical part, we first show that the EDS rule can recover teacher's parameters LIF and LRF neurons (sec. 33.). Subsequently,
we investigate the influence and functionality of the EDS factor and compare it to the SuperSpike surrogate gradient [23] lesion study (sec. 3.4). Finally, we test the robustness of the algorithm to temporal noise in the teacher signal (sec. 3.5
3.1 Gradients
Integrate-and-fire neuron: Without explicit derivation, the partial derivatives of the membrane potential V (eq. B with respect to one of the synaptic weights Wij, the synaptic time constant T., membrane time constant Tm and the reset potential V,, are
av₁(t) Owij
EKAA))
(V,(t)
1
Σ
Wij
Af,
exp
Af:
(9)
θT.
7
Ts
(f.)

---

(V)
N Σ w/j Σ mm
Ad;
1
(-
(- Tm
Aig
Af, exp
+
(V,
Vin)
Σ
Dry exp
(10)
0mm
Tm
mm
[4.)
(t) av,
Ady Tim
Σ {6}
exp
(11)
Resonate-and-fire neuron: Similarly, deriving the partial derivatives for the membrane potential V (cq. of the leaky resonate-and- neuron with respect to single synaptic weight wij, the damping factor b, the frequency of the subthreshold oscillations w and the reset values V, and 7, results in
av1(1) Owij
Σ (121)))
exp(Af,)) (in
(12)
(t)
0b Σ Whj Σ
(f11))
(13)
(V1(1) = Σ Waj Σ
N
Afte(A,,) cos(Atw)) + cos sin((
dw
{,,b,}
(14)
(t) av,
(v,(t) 01,
(15)
and
exp(Ai,b) cos (is()))w)
(16)
3.2 Event-based dynamics
The closed form solution of the membrane potential dynamics and its partial derivatives comprise sums over input and output spike times. These are computationally expensive and create a large memory footprint, because all past spike times have to be stored and summed over. Since the summands are decaying exponential functions, one approximate solution is to only sum over terms that are still contributing significantly. Here, instead relying on an approximation, we show that can replace these sums by factorising the exponentials. As result, the sums over spike times replaced by a recurrence relation, which depends on constants that are updated on each spike event only. Using these event-based dynamics, both the computational as well the memory complexity can be significantly reduced. For all factorised equations and their derivations see appendix
3.3 Joint online learning of synaptic weights and intrinsic parameters
We asked whether it is possible to learn neural computation from sparse temporal feedback. We found that, indeed, the EDS rule can be used to recover teacher's parameters in LIF and LRF neurons for biologically plausible initial and target parameter values (Fig. A-B). The learning dynamics contract but do not converge to the solution i.c., parameter updates fluctuate around the solution without further decay (appendix F). As consequence, the size of the fluctuations and therefore the final precision of the solution Jependent on the learning rate. Since learning happens online, the student can also recover teachers that vary over time (appendix G).
During training, teacher and student output spike trains become more correlated (Fig. (CCE). After training, average of 94% + 3% SE of LIF and 96.3% + 3% SE of LRF output spike times match the teachers' output exactly. For LIF neurons, 3% of the remaining spikes fall within Ims before, and 2.3% within mss after the target time. For LRF neurons, these values are 1.4% and 1.9%. Which, combined, addresses more than 99% of all spikes. Final paramter fluctuations LIF neurons, especially in the synaptic weights, are higher than in LRF neurons, possibly because synaptic currents

---

A
4.0-
2.04
1.0
10-
2.0-
0.5
0.0-
0.0
0.-
2.0
-1.0-
0..
-2.5
0.5
-10-
5.4
-10
10
16 10
10 16'
10
16'
10a
16
16
10 10.
B
Time [min)
[min]
(min)
(min)
5.0-
1.0
5.0
10-
6000
ellowe
0.5
0.0-
0.0
0.0
0.0
0.0
0.0
-0.5
.05-
9.0
-1.0
.5.0
-.0
30th 12 10*
10
12 10
10t 10
10t 10'
12 10"
10 10* [mi]]
10a 10'
[min]
[min]
Time [mih)
Time [min)
Time [min)
1.0
12
Corvengence (mtin
8000
E 1.0-
12 min
Coo Coovengene [mmin)
6000
0.8
120 mn
2000
0.8
120 min
0.-
Rnal
0.5
Final
4000
0.4
02-
2000
0.4
0.2
2000
0.0
100
20
30
10 Rring Rote [Hz]
Splie Offset ms]]
Splke Offset [ms]
Figure 3: Learning dynamics. A. B: Signed relative error (eq. 7 between the students' and teachers' parameters over time. Parameter trajectories converge to and stay within the error margins (grey). A: LIF neuron parameters, from left to right, are: synaptic weights, synaptic time constant, membrane time constant, reset potential and target firing rate and accordingly for B: LRF neurons, synaptic weights, damping factor, frequency of the subthreshold oscillations, reset potential, reset current and target firing rate. C, E: Probability distribution of students' output spike times around target spikes for LIF (C) and LRF (E) neurons after 0.1%, 1% and 100% training time. D, F: Change of convergence time with respect to target firing rates for LIF (D) and LRF (F) neurons.
not reset after output spikes in LIF neurons, making credit assignment more difficult. For both neuron models, the convergence times decrease as the target firing rates increase (Fig. (P.F). This is expected, since parameter updates occur after each missed or false positive spike and therefore higher firing rates result in more parameter updates in the same amount of training time.
Importantly, synaptic weights and output firing rates deviate from their target values and performance significantly inclined when restricting training to synaptic weights. For LIF (and LRF) neurons only 18.5% + 0.7% SE (and 12.9% + 0.6% SE) of output spikes are direct hits and an additional 25.3% (and 19.8%) fall within the Ims window around the target spike, addressing for less than 44% (and 33%) of total spikes (appendix H
3.4 Influence and necessity of the event-dependent scaling factor
If we use the "vanilla" parameter update rule (eq. [ without A(Ad+)) synaptic weights vanish, and the intrinsic parameters diverge or settle at a constant but wrong value, i.c., the training fails (appendix D}. We designed the EDS factor such that parameter updates are not immediately reversed by the next error signal, i.e., consecutive parameter updates are down-scaled as a function of their temporal proximity. To better understand the effect of the EDS factor, we trained parameters separately, as well as in all possible combinations. To this end, we optimised subsets of the student's parameters and kept all other parameters fixed to their optimal, target values. In LIF neurons, single parameters, except the reset potential, can be learned reliably without the EDS factor, whereas optimisation of parameter combinations fail (Fig. [А). Surprisingly, the performance of the SuperSpike surrogate equal to or worse than the vanilla learning rule (except for the pair w, Tm.) This is due to local minima, or degenerate solutions in which the firing rate is kept stable while parameters diverge (appendix K.. rather than larger than permissible noise levels (section 2.6). When optimising synaptic weights only, the recovery rate increases as, through changing B, the surrogate is getting more similar to the vanilla learning rule (Fig. (B). A similar trend can be observed for LRF neurons, where the performance of surrogate gradients falls below the vanilla performance (Fig. FC.D plot with all parameter combinations in appendix D and the performance of surrogate gradients increases the more similar it becomes to and settles at the optimal performance of vanilla learning. Our learning rule performs well for LRF neurons across all parameter combinations. fact, including more parameters is helping in situation where single parameter would get stuck in local minima (data not shown).
8

---

Mondll
160-
Oura
20.0
2- 12--221
H
Ga Tweet
6.24
ms
0.c0-
-0.3
46 m
Liss
-0.2
22-
25 50
25
50
-230
-100 10 -106 l0
-160 10 -106 10 -1ao
Oucaut Rrin Rooz
Ouggut Fring
Gu4pul
Splte Offeetmss]
Spilko offiet [md
Figure 4: Event-dependent scaling factor and temporal noise. A, C: Percentage of 30 students that converged to the target values for all possible parameter combinations with vanilla, surrogate gradient and EDS algorithm LIF (A) and LRF (C) neurons. B, D: Training only synaptic weights with surrogate gradients for varying / values in LIF (B) and LRF (D) neurons. E, F: Expected gradient value of the partial derivative with respect to V, for uncorrelated (E) and correlated (F) Poisson spike trains without (left) and with EDS factor (right). G. H: Probability distribution of students' output spike times around target spikes for three noise levels in the target spike times.
Further insights into why the EDS factor is so effective, can be acquired by analysing expected parameter updates. For example, if the target and output spike trains of LIF neuron are uncorrelated Poisson spike trains, the expected vanilla parameter updates with respect to the reset potential F/ [V, are accurately positive if the output firing rate is too low but negative if the output firing rate is too high (Fig. 4E). However, as training progresses, output and target spike trains become highly correlated. Accordingly, if we calculate E[AV] with the vanilla learning rule for an output-spike train that is a shifted version of the target spike train, i.e. highly correlated, the average parameter updates are distorted (Fig. 4.. left). They linearly increase for positive offsets but stay around zero for negative offsets, leading to inaccurate parameter updates. In contrast, the EDS rule disentangles E| Vr], such that too early spikes lead to positive updates and too late spikes to negative updates of equal size, accurately leading to a respectively reduced or increased reset potential (Fig. 4 right).
3.5 Influence of temporal noise in the teacher signal
Adding temporal, Gaussian noise with standard deviation dg to the target spike times decreases the precision of the solution found by the student as noise levels increase, but does generally not brake the algorithm (Fig. AG.H. However, LRF neurons with target firing rates above 35Hz start to diverge at the highest noise Tevel (data not shown). This is not surprising, when we consider that the average interval at 35Hz is approximately 29ms and temporal noise 0g = 3 frequently yields perturbations >> 5ms, rendering noise indistinguishable from signal. Considering the lower average target firing rates in LRF neurons in comparison to LIF neurons, also explains why higher noise levels yield to, on average, similar precision in LRF comparing to LIF neurons at varying noise levels. More generally, adding temporal noise to the feedback signal reveals subtle relation between firing rates, noise variance and the resulting permissible signal to noise ratio.
4 Discussion
Learning of input-output functions from a continuous stream of information is difficult. We proposed gradient-based update rule capable of learning the unique parameters that determine neural computation from ungraded, unsigned and temporally sparse feedback signals. Importantly, EDS learns online and without backpropagating errors backward in time, i.e., without explicit memory traces of input or target spike times, or past states of the neuron. Instead, we derived an efficient and exact implementation of membrane potential dynamics and their gradients. contrast to previous theoretical work, but in alignment with observations in biological neurons B 4 parameters like time constants etc. as crucial components of learning. Therefore, in line with recent theoretical work suggesting that networks benefit from heterogeneous neural computations [4] 42) we optimise the intrinsic parameters along the synaptic weights, and further, derive the learning rule for two neuron models that implement different computations: the frequently used and optimised LIF
we regard intrinsic

---

neuron and the LRF neuron. Whether EDS can be extended to non-linear models like quadratic 43 or exponential LIF 44 or Izhikevich neurons must be investigated.
Numerical simulations show that our algorithm can learn exact neural computations across a wide range of biologically plausible target values. The EDS rule adapt to changing target computations mid-flight and is robust against temporal noise in the feedback signal. Optimising different combina- tions of parameters revealed the crucial role played by the EDS factor \(Ad*). Surprisingly, the EDS factor cannot be replaced by surrogate gradient. While surrogate gradients are the current state of the art train spiking multi-layer neural networks with backpropagation through time, they may not be optimal to learn neural computations that go beyond rate coding. More specifically, the observation that surrogate gradients produce diverging parameter values but stabilised firing rates suggests that mutual online learning of intrinsic parameters and synaptic weights from target spike times requires a scaling of surrogate gradients.
demendepeddnnt scaling factor in contrast to the
would be interesting, but brake the online learning assumption, to replace surrogate gradients with EDS factors in multi-layer networks and train them with backpropagation through time. An online learning alternative could use EDS to extend difference target propagation [45] 46 from rate-coded neurons to spiking neural networks. In such network, the parameters of neurons that provide the downstream signals are optimised to predict spike times of neurons in lower layers, and parameters of upstream neurons are updated such that the backpropagated target spike times are generated.
4.1 Limitations and conclusions
While we motivate the EDS factor A A(Ad+) by means of its empirical success, a rigorous theoretical explanation for its function is missing. An analytical derivation of the expected gradient values for correlated output and target spike trains in spirit of figure E and F could further unveil the underlying principles of our algorithm.
In LRF neurons target computations with sustained subthreshold oscillations (-b < /mm)) or spike bursts, i.e.. dynamics that reset outside of the separatrix, cannot be learned reliably (data not shown). In addition, target firing rates in LRF neurons remain low, due to the difficulty of achieving high output firing rates from uncorrelated Poisson inputs while keeping balance between excitation and inhibition. If our algorithm also works for correlated, oscillating or bursting inputs near the neuron's 's eigenfrequency, which cause high output firing rates, is left for future studies.
Further, convergence to target values is slow (up to RJ 5.5 and ss 4.3 days of simulated time LIF and LRF neurons respectively). As a proof of principle, we started each simulation at random point in parameter space and required convergence to a second, randomly sampled point in parameter space. This could lead to long learning trajectories and slow convergence times (see also (47), especially when target firing rates were low. In contrast, the parameter subspace that results in good performance in supervised rate-coded networks is large [48], and spiking neural networks may solve a task with a wide range of different architectures or parameters (27]. From a biological perspective, most electrophysiological changes induced by plasticity mechanisms are gradual and complete reorganisation of a neuron's 's electrophysiology (as in our study) implausible. Ultimately, training speed could be increased dramatically (>10x) by increasing learning rates at the cost of losing precision (appendix F) or by using learning rate schedule to speed up initial learning and subsequently gain precision as the learning rate decays. Convergence time could be further reduced by batch-learning and when modelling biological data, by using educated guesses for initial model parameters (c.g. pick parameters such that the initial firing rate of the model and data are identical).
We conclude that the complex temporal dynamics of neural computations can be learned online with a local gradient-basse based update rule from feedback signals that indicate target spike times. Our event- dependent scaling rule will help to bring the success of gradient-base based optimisation to spiking neural networks performing heterogeneous computations and contribute the exploration of gradient- based learning in the brain.
10

---

Acknowledgments and Disclosure of Funding
We would like thank Professor Dr. Henning Sprekeler for his valuable suggestions and Dr. Andrew Saxe, Milan Klöwer and Anna Wallis for their constructive feedback on the manuscript. Lukas Braun was supported by the Network of European Neuroscience Schools through their NENS Exchange Grant program, by the European Union through their European Community Action Scheme for the Mobility of University Students, the Woodward Scholarship awarded by Wadham College, Oxford and the Medical Research Council MMR/N01468// Tim P. Vogels was supported by Wellcome Trust Senior Research Fellowship (21431672/187)].
References
[1] Eugene M Izhikevich. Simple model of spiking neurons. IEEE Transactions on neural networks, 14961755,, 2003.
[2] Ami Citri and Robert C Malenka. Synaptic plasticity: multiple forms, functions, and mecha- nisms. Neuropsychophrrmaccoyy.. 13(1),18-41, 2008.
[3] Dominique Debanne, Gaël Daoudal, Valérie Sourdet, and Michaël Russier. Brain plasticity and ion channels. Journal of Physiology-Parts 97(4-6):403-414, 2003.
[4] Dominique Debanne and Michaël Russier. The contribution of ion channels in input-output plasticity. Neurobiology of learning and memory, 166: 1070959 2019.
[5] Richard Stein. Some models of neuronal variability. Biophysical journal, 7(1):37-68, 1967.
[6] Eugene M Izhikevich. Resonate-and-fie neurons. Neural networks, 14(6-7):883-894 2001.
[7] David E Rumelhart, Geoffrey E Hinton, and Ronald Williams. Learning internal represen- tations by error propagation. Technical report, California Univ San Diego La Jolla Inst for Cognitive Science, 1985.
[8] Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature, 521(7553):436-444 2015.
[9] Jean-Pascal Pfister, Taro Toyoizumi, David Barber, and Wulfram Gerstner. Optimal spike- timing-dependent plasticity for precise action potential firing supervised learning. Neural computation, 18(6):1388-144 2006.
[10] Sander M Bohte, Joost N Kok, and Johannes A La Poutré. Spikeprop: backpropagation for networks of spiking neurons. In ESANN, volume 48, pages 17-37, 2000.
[11] Dongsung Huh and Terrence Sejnowski. Gradient descent for spiking neural networks. arXiv preprint arXiv:1706.0m 04698, 2017.
[12] Filip Ponulak and Andrzej Kasiński. Supervised learning in spiking neural networks with resume: sequence learning, classification, and spike shifting. Neural computation, 22(2): 467-510, 2010.
[13] Räzvan V Florian. The chronotron: A neuron that learns to fire temporally precise spike patterns. PloS one, 7(8)340023, 2012.
[14] Brian Gardner and André Grüning. Supervised learning in spiking neural networks for precise temporal encoding. PloS one, 188))=0061333) 2016.
[15] Timo C Wunderlich and Christian Pehle. Eventprop: Backpropagation for exact gradients in spiking neural networks. arXiv preprint arXiv:2009.08375. 2020.
[16] Larry Abbott, Brian DePasquale, and Raoul-Martin Memmesheimer. Building functional networks of spiking model neurons. Nature neuroscience, 19(3):350-355 2016.
[17] Wieland Brendel, Ralph Bourdoukan, Pietro Vertechi, Christian K Machens, and Sophie Denéve. Learning to represent signals spike by spike. PLoS computational biology, 1603)e10 2020.

---

[18] Eric Hunsberger and Chris Eliasmith. Spiking deep networks with lif neurons. arXiv preprint arXiv:1510.08829, 2015.
[19] Robert Gütig and Haim Sompolinsky. The tempotron: a neuron that learns spike tining-based decisions. Nature neuroscience, 9(3):420-444 428, 2006.
[20] Robert Gütig. Spiking neurons can discover predictive features by aggregate-labe learning. Science, 351(6277), 2016.
[21] Matthieu Courbariaux, Itay Hubara, Daniel Soudry, Ran El- Da-Yaiiv, and Yoshua Bengio. Binarized neural networks: Training deep neural networks with weights and activations constrained to+ or-1. arXiv preprint 1602.00602..200 2016.
[22] Steven K Esser, Paul A Merolla. John V Arthur, Andrew Cassidy, Rathinakumar Appuswamy, Alexander A Andreopoulos, David J Berg, Jeffrey L McKinstry, Timothy Melano, Davis R Barch, al. Convolutional networks for fast, energy-efficiene neuromorphic computing. Proceedings 2016.
of the national academy of sciences, 113(41):11411111)
[23] Friedemann Zenke and Surya Ganguli. Superspike: Supervised learning in multilayer spiking neural networks. Neural computation, 3066):511-1541, 2018.
[24] Paul Werbos. Generalization of backpropagation with application to a recurrent gas market model. Neural networks, 1(4):339-3555 1988.
[25] Guillaume Bellec, Darjan Salaj. Anand Subramoney, Robert Legenstein, and Wolfgang Maass. Long short-term memory and learning-to-len to-lemm in networks of spiking neurons. In Advances in Neural Information Processing Systems, pages 787-797, 2018.
[26] Emre O Neftci, Hesham Mostafa, and Friedemann Zenke. Surrogate gradient learning in spiking neural networks. IEEE Signal Processing Magazine, 36:61-63, 2019.
[27]
Friedemann Zenke and Tim P Vogels. The remarkable robustness of surrogate gradient learning for instilling complex function in spiking neural networks. Neural Computation, 33( -25,,
2021.
[28] Nicolas Perez-Nieves and Dan F. M. Goodman. Sparse spiking gradient descent, 2021.
[29] Raoul-Martin Memmesheimer, Ran Rubin, Bence P Ölveezky, and Haim Sompolinsky. Learning
precisely timed spikes. Neuron, 82(4):925 9988, 2014.
[30] Xiangwen Wang, Xianghong Lin, and Xiaochao Dang. Supervised learning in spiking neural networks: A review of algorithms and evaluations. Neural Networks, 125:258-280. 2020.
[31] Michael Bichl and Holm Schwarze. Learning by online gradient descent. Journal of Physics A, 28:643-655 1995.
[32] David Saad and Sara A Solla. Exact solution for on-line learning in multilayer neural networks. Physical Review Letters, 74(21):4337, 1995.
[33] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint 1rXiv:1412.6980, 2014.
[34] Wulfram Gerstner. Spike-response model. Scholarpedia, 3(12); 13433, 2008.
[35] Bruce Hutcheon and Yosef Yarom. Resonance, oscillation and the intrinsic frequency prefer- ences of neurons. Trends neurosciences, 23(5):216-222 2000.
[36] Tim P Vogels, Henning Sprekeler, Friedemann Zenke, Claudia Clopath, and Wulfram Gerst- ner. Inhibitory plasticity balances excitation and inhibition in sensory pathways and memory networks. Science, 13446066):1566-11 2011.
[37] Sen Song, Per Jesper Sjöström, Markus Reigl, Sacha Nelson, and Dmitri B Chklovskii. Highly nonrandom features of synaptic connectivity in local cortical eircuits. PLoS Biol, 3(3)=66, 2005.
12

---

[38]
Erik De Schutter. Computational modeling methods for neuroscientists. The MIT Press, 2009.
[39]
Henry Dale.
Pharmacology and nerve-endings, 1935.
[40] Gaël Guennebaud, Benoît Jacob, al. Eigen v3. http://eigee..uxfaaiiy. 2010.
[41]
Fleur Zeldenrust, Boris Gutkin, and Sophie Denéve. Efficient and robust coding in heteroge- neous recurrent networks. PLOS Computational Biology. 17(4)cc000667, 2021.
[42] Nicolas Perez-Nieves, Vincent CH Leung, Pier Luigi Dragotti, and Dan FM Goodman. Neural heterogeneity promotes robust learning. bioRxiv, pages 2020-12 12, 2021.
[43] Peter E Latham, BJ Richmond, PG Nelson, and Nirenberg. Intrinsic dynamics in neuronal networks. i. theory. Journal of neurophysiology, 83(2):808-822) 2000.
[44] Romain Brette and Wulfram Gerstner. Adaptive exponential integrate setgrrateannd-iie and-fire model as effective description of neuronal activity. Journal of neurophysiology, 94(5):337-3342, 2005.
[45] Dong-Hyun Lee, Saizheng Zhang, Asja Fischer, and Yoshua Bengio. Difference target propaga- tion. Joint european conference on machine learning and knowledge discovery in databases, pages 498-515. Springer, 2015.
[46] Timothy Lillicrap, Adam Santoro, Luke Marris, Colin Akerman, and Geoffrey Hinton. Backpropagation and the brain. Nature Reviews Neuroscience, 21(6):335- 3366346, 2020.
[47] Kenji Fukumizu and Shun-ichi Amari. Local minima and plateaus in hierarchical structures of multilayer perceptrons. Neural networks, 13(3)3317-377, 2000.
[48] Madhu S Advani, Andrew M Saxe, and Haim Sompolinsky. High-dimensional dynamics of generalization error in neural networks. Neural Networks, 2020.
[49] Gilbert Strang. Differential equations and linear algebra. Wellesley-Cambridge Press Wellesley, 2014.
[50] Mary Natrella et al. Nist/sematech e-handbook of statistical methods. https:///ww. itl. nist. covvkiniSehamtbooo/cdddeet/////d 2014.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes].
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A] No proofs. All derivations are in the appendix.
(b) Did you include complete proofs of all theoretical results? [N/A] No proofs. All derivations in the appendix.
3. If you experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either the supplemental material or as URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes]
(c) Did you report error bars (e-g., with respect the random seed after running experi- ments multiple times)? [Yes] We report standard errors were applicable
13

---

(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes]
4. If you using existing assets (e.g., code, data, models) curating/releasing assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [Yes] See section 2.7
(c) Did you include any new assets either the supplemental material or as URL? [Yes] All code to replicate all experiments and figures
(d) Did you discuss whether and how consent was obtained from people whose data you'le rr using/curating? [N/A] Free and open-source software only
(e) Did you discuss whether the data you are using/curating contains personally identifiable information offensive content? [N/A] No personal data
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid participants and the total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "Zvd6dUqiBxv", "reviewer": "Reviewer_meDX", "summary": "This paper introduces the event-dependent scaling (EDS) rule to use in neural network training that allows neural computations to be learned online - i.e. without error propagation. It presents a teacher-student paradigm for problem formulation, then characterizes the error signal and EDS rule. It then describes its architectures (leaky integrate-and-fire or leaky resonate-and-fire neurons, Poisson-distributed input spike trains, and other standard parameters), and metrics (of convergence and robustness). Next, it discusses resulting learning dynamics and analyses the effectiveness of EDS empirically, and concludes by acknowledging concerns behind lack of theoretical backing and a rarefied testing paradigm.  \n", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "*Originality:* The primary contribution of this paper is an event-dependent scaling online learning rule. To my knowledge, this is a novel and original contribution to the field, building on top of existing work in online learning rules, spiking architectures, and methods of training spiking architectures given that they are nondifferentiable. The introduction cites helpful relevant work to set up the problem, and the related work subsection does both careful and thorough review of existing work in related fields and differentiation from similar work.\n\n*Quality:* I believe this paper is technically sound. The empirical analysis is convincing and the figures are helpful - I don’t feel that the text overstates the effectiveness of the EDS rule. The ablation and gradient analysis in section 3.4 is useful, and reasonable and accurate to the best of my understanding. The dynamics in Fig 3 and various descriptive statistics shown in Fig 4 are effective and say a lot on their own. \n\nThe methods are also detailed and make a lot of sense. I think section 2 is the best section of the paper - it lays out each section of the experimental setup well, highlights the contribution in section 2.3, and keeps all non-novel parts standard and familiar. My only concern with methods is the teacher-student paradigm. It feels very toy because the solution is both single-dimensional and an exact imitation of the teacher. This could easily be a limitation of my own knowledge, but the uses of teacher-student I’ve seen before (and the ones discussed in this paper) are from a very long time ago and generally used for theoretical work where the simplicity of the problem is at least mitigated by the fact that the solution is provably reachable/entailed by the initialization. For an empirical work, it feels like a good but not independent contribution, but again I recognize that this could be a limitation of my knowledge. I will say that the authors discuss this limitation and others with honesty and detail, and the paper seems to warrant its claims.\n\n*Clarity:* The clarity of this paper fluctuates. Section 1 is clearly written, and section 2 is not only clearly written but well-organized, with each subsection answering the question I have or the next thing I want to know at the end of the previous section. I think this is also true of the subsection structure in section 3. \n\nThe writing in Section 3 is less clear, and sounds more like a process statement than results and analysis. There is a lot of “we designed X”, “we trained Y”, etc. This is mainly a stylistic issue, but it does make the organization feel temporal/chronological rather than claims-driven and this leads to a results section that is hard to understand despite holding valuable information. It would help to have some framework at the beginning that outlines the flow of the argument in the section 3 subsections and makes all main claims, let each subsection derive from that framework.\n\n*Significance:* I find this method to be important and unique. The results seem to open a new direction in online learning without backprop that people could in theory expand or extend upon. Insofar as spiking neurons are a good model of the brain, this paper seems to advance their modeling capacity. However, I am not confident that the expansion or extension is actually viable. The problem seems very toy and while it could be a limitation of my own knowledge, I don't know it to be one that has a history of transferring well to more complex domains. \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The limitations are discussed in detail and hit upon all concerns I could think of. My only disagreements with the authors would be on the magnitude of the limitations, not the limitations themselves. The authors don’t really discuss societal impact, and there may be some from accurate models of biological intelligence that can’t be dismissed by saying the work is theoretical, but we are nowhere close so I don’t think it warrants a flag. \n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}, {"review_id": "Ea5qXx1ahD6", "reviewer": "Reviewer_awD2", "summary": "The current paper presented an algorithm using sparse feedback signals at spikes and gradient-based updating rule to train the intrinsic parameter of spiking neuron models. The authors laid out the parameter updating rules for leaky-integrate-and-fire neuron and resonant-and-fire neuron models, and demonstrated the algorithm in training the two neuron types to recover a teacher signal.  ", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "In general, the current work presented a learning algorithm for intrinsic properties of spiking neurons. Following the current learning rule, the parameter update rate was both modulated by the sign of an event (a miss or false positive spike), and also regulated by the proximity of the event. The authors showed that the current parameter updating rule allows various parameters to converge robustly. In contrast, a ‘vanilla’ updating rule (without temporal dependency) or a surrogate rule (instead of temporal dependency, it use voltage dependency rule) both were failed to converge in multiple simulations. The paper was well written and messages were clearly delivered. However, the learning algorithm converges extremely slow on neurons with relatively low firing rate (<10 Hz, >10 h). Notice that this range of firing rate (< 10 Hz) is biologically meaningful. The slow convergence might limited the application of the current algorithm. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors acknowledged that the current parameter updating rule lacks theoretical support. In addition, the current work demonstrated the learning algorithm in a simple teacher-student training paradigm. It is unclear how it would behavior in a more complicated training environment. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "1MUKEcqn6z", "reviewer": "Reviewer_V7xW", "summary": "This work demonstrates that the intrinsic parameters of a leaky integrate-and-fire neuron and a resonate-and-fire neuron can be learned in a student-teacher paradigm. The student neuron learns the intrinsic parameters of the teacher neuron using only the teacher neuron's spike times and gradient based updates.", "questions": "", "limitations": "", "rating": 7, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n\n1. This work demonstrates that the internal parameters of a neuron can be learned using only spike-time feedback. This is important since the neural computations are sensitive to changes in intrinsic parameters. While most ANN models focus on synaptic updates, this work suggests that adapting the internal parameters of a neuron can play an important rule in learning.\n\nWeaknesses:\n\n1. The work seems like an extension of [29] to include internal parameters of a neuron and therefore the results are perhaps not surprising.\n\n2. If the reader is to be convinced that the internal parameters of the neuron are important for learning, then there should be a comparison of the expressivity of a neuron with fixed internal parameters (but adaptive synapses) and a neuron with adaptive internal parameters (and adaptive synapses). \n\n3. The EDS rule is presented without a theoretical justification.\n\nOverall: This work is an interesting proof-of-concept demonstration that the internal parameters of a neuron can be learned and suggests that these parameters should be given more attention.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors address the fact that the EDS rule lacks a theoretical justification; however, the other weaknesses listed above are not addressed.", "review_text": "", "extra_scores": {"ethical_concerns": "I have no ethical concerns.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents an approach for optimizing the intrinsic parameters of spiking neurons, and tests it within a teacher-student paradigm. The initial reviews were generally positive, and reviewers agreed the paper was technically sound and interesting, though there was some concern about the paradigm used for evaluation. However, after the authors' rebuttals, and some discussion amongst reviewers, a consensus was reached that this paper merits acceptance at NeurIPS.
Decision: Accept (Poster)