Spike-timing-dependent Hebbian learning as noisy gradient descent
Authors: Niklas Dexheimer
Sascha Gaudlitz
Johannes Schmidt-Hieber
Venue: NeurIPS 2025 poster ()
Keywords: Biological neural networks
Hebbian learning
Spike-timing-dependent plasticity
Noisy gradient descent
Mirror descent
Primary area: neuroscience_and_cognitive_science
TL;DR: Spike-timing-dependent plasticity can be rephrased as noisy gradient, which allows to obtain convergence guarantees for the stochastic learning dynamics.

Abstract: Hebbian learning is a key principle underlying learning in biological neural networks. We relate a Hebbian spike-timing-dependent plasticity rule to noisy gradient descent with respect to a non-convex loss function on the probability simplex. Despite the constant injection of noise and the non-convexity of the underlying optimization problem, one can rigorously prove that the considered Hebbian learning dynamic identifies the presynaptic neuron with the highest activity and that the convergence is exponentially fast in the number of iterations. This is non-standard and surprising as typically noisy gradient descent with fixed noise level only converges to a stationary regime where the noise causes the dynamic to fluctuate around a minimiser.

Spike-timing-dependent Hebbian learning as noisy gradient descent
Niklas Dexheimer
Saseha Gaudlitz Johannes Schmidt-Hieber'
'University of Twente
Berlin sascha. waw.haagaudiiithhuu-aeeeeee
{n. dexheimer, s.j.schmidtt htwphdtthhieebrreuutte ml
Abstract
Hebbian learning is key principle underlying learning in biological neural net- works. We relate Hebbian tiiittiimmeennnnn plasticity rule to noisy gradi- ent descent with respect to a non-convex loss function on the probability simplex. Despite the constant injection of noise and the non-convexity of the underlying optimization problem, one can rigorously prove that the considered Hebbian learn- ing dynamic identifies the presynaptic neuron with the highest activity and that the convergence is exponentially fast in the number of iterations. This is non-standard and surprising as typically noisy gradient descent with fixed noise level only con- verges to stationary regime where the noise causes the dynamic to fluctuate around minimiser.
Introduction
Hebbian learning is fundamental concept in computational neuroscience, dating back to Hebb [TG]. In this work, provide rigorous analysis of Hebbian spike-timing piike-tining-depnddee dependent plasticity (STDP) rule. Those are learning rules for the synaptic strength parameters that only depend on the spike times of the involved neurons. More precisely, we consider neural network com- posed of d presynaptic/input neurons, which connected one postsynap- tic/output neuron. The presynaptic neurons communicate with the postsynap- tic neuron by sending spike sequences, the so-called spike-trains. Reweighted by synaptic strength parameters w Wa 2 0, they contribute to the post- synaptic membrane potential. Whenever the postsynaptic membrane potential exceeds a threshold, the postsynaptic neuron emits spike, and the membrane potential reset to zero. Experiments have shown the following stylised facts, which lic at the core of Hebbian learning based on spikes: (1) Locality: The change of the synaptic weight w/ depends only on the spike-train of neuron i and the postsynaptic spike-train. (2) Spike-timing: The change of the synaptic weight w; depends on the relative timing of presynaptic spikes of neuron and of the postsynaptic neuron. More precisely, pre-post spike sequence tends increase wi, whereas a post-pre sequence tends to decrease w,. refer to Morrison et [28] for a more comprehensive list of experimental results on STDP rules.
Input
Output
Nenzon
Naun
Figure 1: Neural
network with a sin- gle output neuron
Hebbian learning rules are well-studied instead of the precise timings of pre- and postsynaptic spikes, only the mean firing rates are taken into account. These rate-based models exhibit many desirable properties, including performing streaming PCA [23, [Р] and receptive field development [I3, Section 11.1.4]. Much less is known if the precise timing of pre- and postsynaptic spikes is considered, since the intrinsic randomness of the dynamics complicates the mathematical analysis.
'These authors contributed equally to this work.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Our main contribution lies in connecting STDP to noisy gradient descent and providing a rigorous convergence analysis of the noisy learning scheme. To this end, we introduce learning rule for the weights wi...., W.d which captures the locality and spike-time dependence of Hebbian STDP. We rewrite the learning rule as a noisy gradient descent scheme with respect to a suitable loss function. The connection to noisy gradient descent and stochastic approximation [ [9]] paves the way for applying mathematical tools from stochastic process theory to analyse the STDP rule. Our analysis of STDP is inspired by the work on noisy gradient descent for non-convex loss functions of Mer- tikopoulos et al. [27]]. By refining their arguments and carefully tracking the error terms, we show an exponentially fast aligmment of the output neuron with the input neuron of the highest mean firing rate on an event of high probability. The specialisation of the output neuron to the input neuron of the
highest intensity is related to the winner-take all mechanism in decision making [I22 39.
[I 25 B[T]
The competitive nature of Hebbian STDP has been observed by [59 68. [5] and the specialisation
to few input neurons is important for receptive field development [*] By connecting Hebbian STDP
to noisy gradient descent, we are able to provide a mathematical analysis beyond ensemble averages and to quantify the speed of convergence.
Taking into account the intrinsic geometry of the probability simplex, we also relate our learning rule to noisy mirror descent, more precisely to noisy entropic gradient descent, which has been proposed for brain-like like learning by Neumann et al. [30], Cornford et al. [IO].
The key contributions are:
1. STDP as noisy gradient descent. We deduce a new framework, in which Hebbian STDP is interpreted as noisy gradient descent. This connection allows to employ powerful tools from the theory of stochastic processes for analysing Hebbian STDP.
2. Linear convergence. We prove the alignment of the output neuron with the input neuron of highest intensity at exponential rate event of high probability.
3. Connection to noisy mirror descent. We relate our learning rule noisy mirror descent, more specifically to entropic gradient descent. This connection facilitates the integration of techniques from both areas, potentially leading to future synergistic effects.
Related literature
Common approaches to understanding STDP restrict to the mean behaviour after taking the en- semble average, e-g. [IK 22 [15],, or compute the full distribution using the master equation of the Markov process [I13, Section 11.2.4]. Unfortunately, the latter is only feasible in specific scenarios. In [2I]], the authors consider general noisy spike-time dependent dynamic which is transformed into a deterministic ODE by imposing a slow learning rate and using the self-averaging effect of the system. A stability analysis reveals structure formation and output stabilisation. One major differ- ence to our work is the influence of the noise. In [2I]], the variance of the weights grows linearly and a careful comparison of time scales is required. In our work, despite a constant injection of noise into the system, the dynamic for the spike-trigeering probabilities converges to a deterministic limit. Secondly, the use of recent ideas from the analysis of noisy SGD allows us to track the influence of the realised noise in every step. A considerable number of previous works derived STDP rules based on the minimisation of a loss function, typically corresponding to the mininization/maxiiizati of some notion of energy or information, see [8. R. D. E2] 42 32, A5 [6] While this approach is appealing, the mathematical analysis of these learning rules is challenging due to the modifications required to achieve biological plausibility. In contrast, we start with a biologically plausible learning rule and utilise the arising loss function to derive mathematical convergence guarantees of the learn- ing rule. The importance of the choice of suitable metric for the derivation of the learning rule is laid out in [А We refer to [0. A8. 67 IE, 25] and the references therein for further results on STDP. In [20], related learning rule is mathematically analyzed by relating it to expert aggregation.
1.1 Notation
Linear algebra. For positive integer d, we write [d] := {1, ... d) and 1 = 11) For i E [d] we denote by e; the 2h standard basis vector of Edd The Hadamard product between two vectors a, b E Rd is denoted by a @ b == (aib)...., abba) E Rd. We write I E Rdxd for the Σd
T)T E Rd.
identity matrix on d and ||ll|2 -== uf for the squared Euclidean norm of vector Rd
2

---

Probability. p) denotes the multinomial distribution with one trial (n 1) and probability vector = (P1,-..., that is 5 M(1,p) if only only IP(g = i) = ps for any E [d]. We denote by
d P == (={DER': p. 2 20 [d].
Ep=11
the probability simplex in R. We denote by 1 the indicator function of set A.
1.2 Hebbian inspired learning rule
Inspired by Hebbian learning, we consider an unsupervised learning dynamic with input (or presy- naptic) neurons and one output (or postsynaptic) neuron. The 7th input neuron has meun firing rate A, > 0 describing the expected number of spikes per time unit. The vector À = .A Ad) contains the d mean firing rates. The strength of the connection between the 7thh input neuron and the output neuron is modulated by the weight parameter w, 2 0, and changes to encode the information of the input firing rates.
We introduce a Hebbian STDP rule Subsection 2 and show that, under some assumptions on the spike-trains, it is equivalent to the following dynamics. If w (()) = (wi (O)...., ......) are the d weights at initialisation, the updating rule from w w(k) to w(k 1) is given by
((k: + 1) = w(k) @ (1 + (B))) + Z())))
where > 0 is the learning rate and k 0, 1.... denotes the postsynaptic spike time. The d- dimensional vector (kk) is the standard basis vector pointing to the presynaptic neuron, which triggered the (A: + 1)th postsynaptic spike. It is given as B(k) = E 1 the one-hot encod- ing of independent multinomial random variables ÇK M(1 (k)). with k-dependent probability vector
À O (k)) A ((k)
pkk)
E ER. k 0
(2)
Since the probabilities (k) model the probability that the (I: + 1)th postsynaptic spike is triggered by neuron = 1,.... ......,d d. we call them (postsynaptic) spike-triggering ceneerrrgggeiigrrbbiiiee probabilities. The i.i.d. d- dimensional vectors Z(k). k 0, model the contribution of presynaptic spikes, which did not trigger the (k: + 1)th postsynaptic spike, to the weight change. They are modelled to have i.i.d. components Zı (k),..... ..... (k), which are supported in [-(Q 1), (Q 1)], for some Q > 1, and centred such that [[[[ - 0.
In the remainder of the paper, we analyse the long-run behaviour of p(k) as h + oo under the learning rule Eq. (.)) We say that the output neuron aligns with the jth input neuron p,(k) + 1 as k > xo. Since the input intensities λ1, ....... da > 0 are fixed throughout the dynamic, this condition is equivalent to (k))k) Σi= (k)k)
+ Figure visualises the learning rule Eq. (2).
2 Representation as noisy gradient descent
We continue by relating the learning rule Eq. (I) to noisy gradient descent. For notational simplicity, define (k) = B(k) Z(k) for k 0, l,.... Combining the weight updates Eq. (II with the formula for the probabilities from Eq. (L), we find
p + 1) = A o (w) o (1+ (I)))
(w(k) o (1 + ( = p(k) @ +Y)())) p(A) (1 (1+)Y(k))
0
(3)
The normalisation in the denominator and the multiplicative nature of the update ensures that the dynamic of p(k) is restricted to the probability simplex. By a Taylor expansion around = 0, we find
p(k + 1) = p(k) @ (1 (1+ (Y p(k) T (()1)) + O
Since
E ( - p(k) T Y()1 p(k)] = pkk) - [|
the random vectors
= :(k)=p(k)) 3 (k) p(k) Y
TY) - p(k) [p)))| == 0.
3

---

990 10--83AAAAM
990 1BA,27 A331
290-1.322.3144
-0.03
2.00
Figure 2: Contour plot of the loss function L from Eq. ( the probability simplex Q for d with different overlays. Left: Three sample trajectories of Eq. (E) with different initial configurations P(0). Middle: Stream plot of the gradient field given by Eq. (Z)). Right: 100 sample trajectories
of Eq. (+) with p0) (0.3, 0.3, 0.4) All trajectories simulated with 2000 iteration steps, learning rate = 0.01 and (k))
1,,
centred. The distribution of {(k) depends w(k) and p(k). Up O(a²) (aaa?)-ttrms,, write the learning rule Eq. (L) as a noisy gradient descent scheme
p(k p(k) (1 ((p(k) (Ip(k) o(() = p(k) - aVL(p(k) + ag(),, == 0, 1,...
(5)
for the loss function
L(p) =
p
2
p p) lpl.
peR R
(6)
1=1
1=1
with gradient
VL = - (p- 1p-11p1=11) E PER
(7)
Dropping Oaa²) terms is only done for illustrative purposes. Our main result (Theorem 2 applies to the original learning rule Eq. (G). The subsequent lemma summarises the key properties of the loss function L from Eq. (6). For = 3. Figure 2 visualises the loss function L and the learning dynamics Eq. (G).
b E tj for some S C [d]. Every critical point with [S] 2 2 is a saddle point. The local minima of the loss function L from Eq. (G) are the standard basis vectors {el,..., ea}. Furthermore, every local minimum of L is also global minimum.
Lemma 2.1. All critical points of the loss function Eq. (B) can be written as p =
2.1 Linear convergence of the learning rule
We state the convergence guarantee for the learning rule Eq. (.) Renaming the indices, we can assume that Pı (0) is the largest initial probability. Provided that Pr is strictly larger than each other component of (0). the following theorem shows linear convergence of the first component to 1 on an event θ in expectation. The probability of θ can be chosen arbitrarily close to by reducing the learning rate a.
Theorem 2.2. Given (0, 1), assume
A pi (0) max p: (0) > 0 and 0 0<a5 16Q2 (11 (1-Qa)*
2
256(1 Pı (0)) (4 + )))
1
i=.....d
Then there exists an event with probability 2 1 - c/2 such that
[[]] - elll < 2 - pi)))) exp
A
12)Rk)
for all 0, I,...
16 d
Consequently, given 8 > 0.
(((((()) eilli 2 26) Se for all
2
16d
log
4(1 P1 (0))
aA (4 dA)
ed

---

Remark 2.3. If all weights are equal at the starting point of the learning algorithm, the assumption
(i( ma. Pi (0) > O, is equivalent to requiring X (0) 2 ), 0. In this case, the convergence of p(k) to corresponds to the network performing a winner-take-all mechanism [122 99 BI, AI, 25J. The competitive selection of input neurons in Hebbian STDP has been observed by [59, GR, [. among others. Our results extend existing findings by going beyond ensemble averages and also provide a rate of convergence for the random dynamics.
The convergence on an event of high probability is in line with other recent results on noisy/stochastic gradient descent for non-convex loss functions, see e.g. [27 [6] or [ا Theorem 2.5]. Contrary to these results, we can choose a constant learning rate and obtain linear convergence. To illustrate the reason for this, we give brief overview of the proof of Theorem 2.2. The full proof can be found in Section A.
1. We restrict the analysis to the event + on which
pi (k)
max 2
pe 2 c> 0,
holds for all iterates k. On this event, the derivative of the first component can be bounded from below by
pi (k)) (R) [Ip())) 2 1 11-pp (k:).
(8)
2. As described in Eq. (L), we apply a Taylor approximation to the original dynamics. We bound the error term for the 5th component (k + 1) by the order a (A) pc(k)). The approximation error is dominated by the gradient update on θ if the learning rate is small enough (see Eq. (.).
3. Similarly as in Eq. (D), we restate the learning increments of the dynamics as the sum of the true gradient and centred noise vector {(})). By Eq. (8), this decomposition yields linear convergence of Pı (k) + on 6.
4. To find lower bound for the probability of the chosen event O we employ a similar strategy as Mertikopoulos et al. [27]]. Through the representation Eq. (D), we can show that o occurs, as soon as M(k)= == a Σt E(()) is uniformly bounded by some sufficiently small constant. As (M())) is a martingale, the probability of the latter event can be controlled through Doob's 'ss submartingale inequality (see Eq. (22)).
5. To apply Doob's ss submartingale inequality, we bound the second moment of M(k). Since the variance of the components of {(k) is also dominated by 1 1-po 1-P1(Ae -P(kk, (k), we achieve a bound of the order a E (1-pi) pı (i)) This series is summable we have linear convergence to O, which allows us to choose a constant learning rate a.
2.2 Associated gradient flow
In this subsection, we consider the associated gradient flow of probabilities p(() as a vector-valued function, which solves the ODE
d
dt (()) = p(1) o (p(1) - (I)())))== -VV(p(()) 12 0
=
and is initialized by the probability vector p(0). By definition, d E p:(t) 0. such that S.Em1:(( = E (0) 1. Since the updating rule is multiplicative, the gradient flow pro- duces for all t 2 0 probability vector. The gradient flow Eq. (D) also occurs as specific replicator equation in evolutionary game theory, see Hofbauer and Sigmund [177 Chapter 7]. Elementary prop- erties and an explicit solution for the gradient flow with d = 2 are derived in Section 2.21. Although the loss function ((p) in Eq. (G) does not satisfy a global Polyak-ELojssiwwic condition, in particular is not globally convex, we can deduce the following convergence for the ODE Eq. (
Theorem 2.4. Assume
: (0)))= Z max (0) + +A,
for some A > O. Then
A (1 [1 (d 1)A)t)
le - p(t)li s 2( - (0)) exp
that is linear convergence of p(t + e1 as t xc.
5

---

2.3 Biological plausibility of the proposed learning rule
We study a biological neural network consisting of d in- put (or presynaptic) neurons, which are connected to one output (or postsynaptic) neuron. For the subsequent argu- ment, we assume that the spike times of the d input neu- rons are given by the corresponding jump times of d in- dependent Poisson processes (X, (x(1) (x(o))20..... )izu..... (x("))2
with respective intensities λ1,... Ag- All neurons are excitatory and each connection between an input neuron j E [d] and the output neuron has a time varying and non-
75
11.1....
negative synaptic strength parameter, which we denote by (t) 2 0.
An idealized model is that spike of the 7th imput neuron at time causes an exponentially decaying contribution to the postsynaptic membrane potential of the form / + wy (r))Ce-((-77)111
Input Nenrons
Output Neuron
We set the parameters C' > 0 to one, as this can always be achieved by a time change + to and change of units of the voltage in the membrane potential.
Figure 3: Considered biological neu- ral network with spike trains and mem- brane potential Y, of the postsynaptic neuron.
If T denotes the spike times of neuron j E [d]. the post- synaptic membrane potential (Y))120 given by Yo =0
and Y, = E (1-(r)e((t-)) for all 2 0 until Y; 2 S, where S > 0 is given threshold value. Once the threshold 5 is surpassed, the postsynaptic neuron emits a spike and its membrane potential reset to rest value, which we assume to 0. Afterwards, the incoming spikes will contribute to rebuilding the postsynaptic membrane potential. If to == 0 < t1 < 12 < denote the postsynaptic spike times, the membrane potential at arbitrary time is therefore given by
d Σ (((())-(1-7) for all 44 < t 4+11
11
Σ
(10)
S=IrEnnn(te.)
We consider the following pair-based spike-timing-dependen plasticity (STDP) rule (I14, Section 19.2.2]): A spike of the jth presynaptic neuron at time causes the weight parameter function t + (t) to decrease at T by (r-(r- where t. is the last postsynaptic spike time before 7 and to increase at any postsynaptic spike time & by a ErTTn(tt (7-1(2) with >> O the learning rate. As common in the literature, spike times that occurred before & only have minor influence and are neglected in the updating of the weights after fg. The term EeeTnn( (r-4) is then the trace (IE), Equation (19.12)]) of the jth presynaptic neuron at time fx+1-
For mathematical convenience, we will assume that all weight-updates in (4x, t+1 are delayed to the postsynaptic spike times 4g in the sense that the learning rule becomes
11)(++1) = wyy(t) (1 [1aa
Σ
e e-(1++1-2 -e
k= 0,
(11)
rETGN(LL.R11)
The postsynaptic spike times ts are the moments at which the postsynaptic membrane potential Y, reaches the threshold S. They depend on the presynaptic spike times. however, the exact dependence hard characterise in the assumed model. For mathematical tractability, we will instead work with an adjusted rule to select the postsynaptic spike times (1, l2.... Since Y, only increases at the presynaptic spike times, 4++1 has to happen at a presynaptic spike time. Denote by Ty1: T.2. the spike times of the j-th presynaptic neuron after the previous postsynaptic spike time & in increasing order. The distribution of 4 4t. is completely determined by the probabilities
Pt Tjæ) P(u Tye [++1 (mmmmm)))P((t
(Tjm)m21) j = 1,..., i 1,...
Based on probabilistic arguments related to the underlying Poisson processes that we outline in Section A.3, we replace the probabilities P(4+1 E (Tim)m21) by the probabilities
Xy1)(te))
(12)
EL (ews

---

Working with Eq. (1) instead of P( E (Tim)m21) results in an approximation of the distribu- tion of 1 Lemma B.7 describes a setting, where Eq. (122) is exact. If all weights are much larger than the threshold S, every presynaptic spike causes a postsynaptic spike. The proof of Lemma A7 can be adapted to this case to show that the probability that the jth neuron emits the first spike is / ///Ei1 mation describes the dynamic the beginning of the learning process. This view is corroborated by experimental results, see point (vi) of Morrison et al. [28, Section 2.1].
f-1 λe. Since Hebbian learning is intrinsically unstable, we argue that the proposed approxi-
Compared to the original definition of f1+1. the proposed sampling scheme has the advantage that the presynaptic spike times, which were not selected as postsynaptic spike time, add centred noise to the updates. More precisely, one can show that by the construction of 4 and the properties of the underlying Poisson processes, the conditional distribution T|{T E (tz, 11-1)3 is uniformly
distributed on (4x. f.+1). By the symmetry relation e (bb-w) e -u(-a) = - -e -ee-(b-)) e(-==)) E [-1,1], which holds for all real numbers a S S b with b + conditionally E (tx, 1e+1), the random variable (1++1-7) -e (r-ta) ta) is centred and supported on [-1, I]. The update rule Eq. (III) then becomes
[a, b]. this implies that
wy (1x-1) = wy)()) (1 + (1) (1-
eth cl1 +a
E Z(..)).
(13)
with centred random variables [r, satisfying [T(:))] S 1. Assuming that the postsynaptic firing rate slow compared to the learning dynamic, we discard the term gle-te+1 < 1. Since
j (I: + 1) follows a multinomial distribution with parameters
)ews((t)). the term
1 (1=-)(((++11)) corresponds to the jth component of B(k) in Eq. (II. This motivates the learning rule Eq. (I). Additional details on the derivation are given in Subsection A.3 of the supplementary material.
3 A mirror descent perspective
In this section, we rewrite the gradient flow Eq. (5) natural gradient descent on the probability simplex and relate the discrete-time learning rule Eq. (E) for the probabilities p to noisy mirror gradient descent.
Recall from Eq. (5)) that the learning rule Eq. (G) can be interpreted as noisy gradient descent with respect to the loss function L from Eq. (K) in the Euclidean geometry. As consider flow probability vectors, a different perspective is to use the natural geometry of the probability simplex. To this end, consider the interior of the probability simplex M == int({F)
a Riemannian manifold with tangent space TpM = {x E R 1 0} for every p E MM. A natural metric on M is given by the Fisher information metric/ Shahshahani metric [L | which is induced by the metric tensor dp: : TpMXX Tp.M + I, (u, v) u diag(p) 'v at M. Here, diag(p) E E is the diagonal matrix with diagonal entries given by p. We refer to Figure I of Mertikopoulos and Sandholm [26] for illustration of unit balls in this metric. The (Riemannian) gradient of the loss function (pp) = - ||||/// /2 with respect to dp is given by
Vd [p) = diag(p) vi(p) E TpM,
where denote by VL the Euclidean gradient of İ. The Riemannian gradient flow is called natural gradient flow in information geometry [3] and Shahshahani gradient flow in evolutionary game theory [[X]]_ When transforming the Euclidean gradient flow for L to Riemannian gradient flow on the probability simplex, the part + ||||/21 is orthogonal to M. Consequently, it does not contribute a direction the probability simplex. Consequently, the Riemannian gradient flow of L and the Euclidean gradient flow L coincide.
The mirror descent algorithm
29] prescribes the discrete-time optimisation algorithm
- (..(..(())
pkk + 1) E argmin PEM
p T V(((())) +
= =,0,,,.. 0,1,...,
(14)
where f: M + is the function to be minimised and क: M x M + RR a suitable proximity function. Euclidean gradient descent is recovered by the choice 4 p(k)) |lp 22 It
7

---

is well-known that the natural gradient flow is the continuous-time analogue of the exponentiated gradient descent or entropic mirror descent, where 4
KL(pllp(k)) is chosen as the Consequently, the gradient flow Eq. H can also be viewed as continuous-time version of entropic mirror descent with respect to f = This connection transfers to the discrete-tima time and noisy updating rule Eq. (G). An alternative approach for connecting our proposed discrete-timg learning rule Eq. (Gi to entropic mirror descent is included Subsection A.4 the supplementary material.
(p.
(k)
Kullback-Leibler divergence between p and pkk) [
97, B5
4 Multiple weight vectors
The learning rule Eq. (1) aligns the output neuron with the input neuron of the highest intensity, but information about the remaining input neurons is unveiled. As a proof-of-concept, we generalise the learning algorithm Eq. (II) estimate the order of the intensities A.,.... Ad. To this end, consider d different output neurons, which are connected to the d input neurons via the
weight vectors Wi..... Wa E Rdd. The weights time k are combined into the matrix
(kk [ (l) wa(k)] E dxd =
Inpert
Output
Norrons
Noumas
and the corresponding probabilities Pt... Pi..... are combined into the matrix P(k) (p)
Figure 4: Neural network with d in- put/output neurons.
pa(k)] E Fd By reordering the neurons, we can achieve λ X2 < ... < Aa. If the intensities are strictly ordered, our goal the alignment of the jth output neuron with the j"h input neuron, which amounts to the convergence of P(k) to the identity matrix E Rddd as time increases. If multiple intensities are equal, convergence up to permutations within the group of equal intensities. We propose Algorithm M. which constitutes an STDP rule as lines 3 4 can be implemented using the spike-trains and the learning rule Eq. (II).
Algorithm 1: Aligning multiple output neurons
Input: K N: number of iterations, (()) e Rdddd weight initialisation, o1,..., Odi
learning rates of the output neurons.
for
for 0,
1....... K do d do
j=
Receive (k) (1,, with (b( X o (xkKAAAT wy(k) and
(k) Unif([-1, 1J4) from spike trains;
4
Compute the base change (w,(k) + (k) @ (By(k) + (k)); Update
(k)) (i(k)
w, 1)
[w; w; wk();;
6
end end
Output: The weight evolution W(k) |w] (k) wa(k)], k ....., K and probability cvolution Pkk) = [p (k) ... (kk), 1, I....,K.
Algorithm M is inspired by Sanger's rule [35] for learning d principal components in streaming PCA. The first weight vector (k) aligns with e by Theorem 2.2 since its dynamic equals the learning rule Eq. (II) By removing the components of the change Aw, (k:) in the direction of wi wi(k).... w wk-1(k) in line 5 of Algorithm M. the weight vector (k) is forced to converge to ej. similarly the Gram-Schmidt algorithm.
Simulations of the corresponding probability matrix P(k) with varying learning rates and drawn i.i.d. from Unif(-11, 1]d) are included in Figure 5. We choose different learning rates for the d vec- tors satisfying (1 > ... a. > 0. This ordering ensures fast convergence of the lower order weight vectors the correct standard basis vector and counteracts the impact of initial misalignments of the higher order weight vectors. The simulation of Algorithm M shown in Figure El displays decrease
8

---

1.5
00-
1.2
06-
1.0
04-
8.0
0.2
155
20
15
2.0
40
mensions d 3. The weights are initialised equally, and the intensities are given by X (10, (75,5))_ 5) The resulting initial probabilities are Pı (0) = P2(0) = Pa(0) = (4/9, 2/99 Left: A single trajectory with learning rates (1, 0.5) and 4 x 10 iterations. The mark- ers and correspond to the probabilities at A: 4 x 10 and k = 10a, Middle & right: The Frobenius error [P(k) - II|2/2 /2 of 100 trajectories with learning rates 10 10-3 (1, 0.75. 0.5) and Figure 5: Probability matrix P(k) arising from the weight dynamic (k) of Algorithm D for di-
10-"(1, 0.75,, 0.5) T respectively.
of the Frobenius enror [P())) 11 over the iteration index k, when averaged (blue line). Never- theless, we observe that for single trajectory, the error can plateau around and 2. Given that the probability vectors tend to converge standard basis vectors {el, ................................ ex). ). non- non-vanishing error is due to an incorrect ordering or duplicates. Consequently, the error [P - -1|2/2 corresponds to the number of output neurons aligning with the incorrect input neuron, and plateaus at 1, and can arise. Theorem 2 shows that this phenomenon can be mitigated by slower learning rates, which is corroborated by decreasing the base learning rate from 10 to 10 in the simulation. A rigorous mathematical analysis of the Algorithm M is challenging due to joint updates all read-out neurons. In Section A.5 we show that Theorem 2 is applicable if the learning is split into disjoint learning periods, and we derive theoretical convergence guarantees.
5 Extensions, discussion and limitations
fime-inhhmooeenous intensities. We considered input spike trains generated from Poisson point processes with fixed intensity. It is natural to extend this to time-inhomogeneous intensities. Here assume that the intensities of the input neurons are constant on the interval (k, k+1] and are stored in the vector (k)). The intensities
A(k) and weights w w(k) determine the spike- triggering-probabiie
p(k) = (kk) @ (k)/A(() w (k: = (A)) and the update formula Eq. (2) becomes
o X(k + 1) @ (w(k) 0 (1 +YY())) O (1++a
plk + 1)
= (15)
A(k: + 1) (w(k) o (1+a))()))
p(k) (1 (1+aY(A))
with ((k) = A(k 1) o w(k)/(X( 1) w(k).. first order Taylor expansion yields
p(k + 1) = (()) の (1 a1+a p(k) ((()1)) + +Oaa)).
(16)
Since E{)(})) = pkk), this means that
pkk + = 3 1 (p(k))()) p(k) ((k)1)) ((k))((())))))
centered noise + O(a²).
(17)
Extending the gradient flow derivation to the time- aemeinnhooggneouss case, one can identify the ODE
d
log (A(1) + p(1) -P() ( log (A(1) + +p(0)
गे p(1) = p(1) @
(18)
dt
where the logarithm is taken componentwise, as corresponding deterministic dynamic in continuous time, Section A.6 for derivation. The ODE be interpreted replicator equation with (time-varying) fitness मे log(A(()) + p(). see e.g. Chapter 7 of [I8). An interesting scenario which lies beyond our mathematical analysis amounts to considering time-dependent mean firing rates that piecewise constant, corresponding the successive exposition to different input patterns [X]
d

---

Correlated inputs, Correlated inputs facilitate simultaneous spiking of different imput spikes. The probability that input i and j spike at the same time is denoted by T and naturally Fú := 1, for all i (d]. We introduce the d x d random symmetric matrix C(k) = (Cig(k)) with independently sampled entries
(k,a()= =- Ck.(k) ~ Ber(T for all i iSje E [d].
C(k) describes the simultaneous spiking of the different inputs at the k-th post-synaptic spike, i.e. if Oit (k) is I then inputs i and both spike at the k-th post-synaptic spike if either i or j caused the post-synaptic spike. Compared to the original model, the random vector Z(k) remains the same, but the random vector B(k) (B₂ (k), (k),.... (a( that encapsulates which of the presynaptic neurons caused the postsynaptic spike is replaced by S(k) C()) S(k) encodes which of the inputs spike at a post-synaptic spike, and in particular it holds
((S.(k) = 1(p(k)) r., (k)) for all E [d],
where (k) denotes the i-th row of r(k). Since the only change in the dynamic of p(k) is replac- ing (k) by S(k) in the definition of Y (k), Eq. (H) still holds true, and we obtain
E[p(k + [)(]())] E p ()) o (1 (1+ a (Y()) (p(#)) (()))) (((s)
(k) + (1+a) (Tp(k) (D(K)) To()))))(()))))
which induces the following gradient flow
d de D(t) p(t) (Tp(t) - (D(t)) Ip(t)1).
(19)
This is again replicator equation with fitness Pp, compare Section 7 of [T]]. The associated Shahshahani-loss is given by x + - Tx. Thus, in the correlated model, the probabilities p follow a flow restricted to the probability simplex aimed at maximising the quadratic form associated to the matrix r. This property is similar to principal component analysis (PCA), as the goal there is to recover the eigenvector corresponding matrix. Thus, the behaviour of the correlated model can be interpreted as a form of PCA restricted to the probability simplex. Theorem A in the supplementary material generalizes Theorem 2/2 to weakly correlated input neurons and is accompanied by simulations in Figure i
the largest eigenvalue of the underlying covariance
Small biological neural network. In this paper, we mathematically analyse the convergence be- haviour of small biological neural network with one layer composed of excitatory presynap- tic/input neurons and multiple postsynaptic/output neurons. It is natural to generalise the setting to account for inhibitory neurons and more than one layer.
Weight explosion. The learning rule Eq. (I) for the weights w(k) causes them to increase without bound as the iteration index l increases. When the weights exceed the spike threshold, the model becomes biologically implausible and the derivation of the probabilities Eq. (I2) is no longer valid. This unstable nature well-known to be intrinsic to Hebbian learning algorithms and is commonly countered by soft or hard bounds, or by including mean-reverting terms to the dynamic Gerstner et al. [14, pages 497-498]. We follow a different route, namely viewing Hebbian learning as a temporal phase of limited length, which is followed by stabilising homeostatic learning phase. This view is corroborated by experimental results, compare Point (vi) in Morrison et al. [28, Section 2.1].
Beyond Pair-based STDP rules. While pair-based learning rules such as Eq. (I)) only account for the relative timing of one pre- and one postsynaptic spike time, also the voltage the location of the synapse should be taken into account [B7]. A natural generalization of our framework would be to extend the results to the model proposed in [S]]
Acknowledgments and Disclosure of Funding
All authors acknowledge support from ERC grant A2B (grant agreement no. 101124751) S. G. has been partially funded by the Deutsche Fonschungsgemeinschatt (DFG, German Research Foun- dation) CRC/TRR 388 "Rough Analysis, Stochastic Dynamics and Related Fields Project ID 516748464, Project B07. Parts of the research were carried out while the authors visited the Si- mons Institute in Berkeley. We thank Xiangyuan Li for pointing out some typos.
10

---

References
[1] L. Abbott and S. Song. Tempporlly asymmetric Hebbian learning, spike liming and neural response variability. In M. Kearns, S. Solla, and D. Cohn, editors, Advances in Neural Infor- mation Processing Systems, volume 11. MIT Press, 1998.
[2] F. Alvarez, J. Bolte, and O. Brahic. Hessian Riemannian gradient flows in convex programming. SIAM Journal on Control and Optimization, 43(2):477-501, 2004.
[3] S.-i. Amari, Natural gradient works efficiently in learning. Neural Computation, 10(2):251- 276, 1998.
[4] S.-i. Amari and Nagaoka. Methods of Information Geometry. Translations of Mathematical Monographs. American Mathematical Society, 2007.
[5] A. Beck and M. Teboulle. Mirror descent and nonlinear projected subgradient methods for convex optimization. Operations Research Letters, 31(3):167-175. 2003.
[6] A. Bell and L. Parra. Maximising sensitivity in a spiking network. In L. Saul, Y. Weiss, and
L. Bottou, editors, Advances in Neural Information Processing Systems, volume 17. MIT Press, 2004.
[7] S. Bohte and M. C. Mozer. Reducing spike train variability: A computational theory of spike- timing dependent plasticity. In L. Saul, Y. Weiss, and L. Bottou, editors, Advances in Neural Information Processing Systems, volume 17. MIT Press, 2004.
[8] G. Chechik. Boike-Timing-Deppnnee Plasticity and Relevant Mutual Information Maximiza- tion. Neural Computation, 15(7):1481-135 2003.
[9] C. Clopath, L. Büsing, E. Vasilaki, and W. Gerstner. voltage-based STDP with homeostasis. Nature Neuroscience, 13(3):344552, 2010.
Connectivity reflects coding: A model of
[10] J. Cornford, R. Pogodin, A. Ghosh, K. Sheng, B. A. Bicknell, O. Codol, B. A. Clark, G. Lajoie, and B. A. Richards. Brain-like learning with exponentiated gradients, 2024. URL http:
//biorxiv.co wttp:///ooooo/o///111110000000/0
10.25.6..... 22.2277...
[11] S. Dereich and Jentzen. Convergence rates for the Adam optimizer, 2024. URL ittps: //arxiv. org/ herggaab//00 abs/2407.210000
[12] J. Feldman and D. Ballard. Connectionist models and their properties. Cognitive Science, 6
(3):205-254, 1982.
[13] W. Gerstner and W. M. Kistler. Spiking Neuron Models: Single Neurons, Populations, Plastic- ity. Cambridge University Press, 2002.
[14] W. Gerstner, W. M. Kistler, R. Naud, and L. Paninski. Neuronal Dynamics: From Single Neurons to Networks and Models of Cognition. Cambridge University Press, 2014.
[15] M. Gilson, A. N. Burkitt, D. B. Grayden, D. A. Thomas, and L. Van Hemmen. Represen- tation of input structure in synaptic weights by spike-timing timing-dependen plasticity. Physical Review E, 82(2):021912, 2010.
[16] D. O. Hebb. The Organization of Behavior. Wiley, 1949.
[17] J. Hofbauer and K. Sigmund. Evolutionary Games and Population Dynamics. Cambridge University Press, 1998.
[18] J. Hofbauer and K. Sigmund. Evolutionary game dynamics. Bulletin of the American Mathe- matical Society, 40(4):47--51, 2003.
[19] D. Huang, Niles Weed, and R. Ward. Streaming k- k-PCA: Efficient guarantees for Ojas algo- rithm, beyond rank-one updates. In M. Belkin and S. Kpotufe, editors, Proceedings of Thirty Fourth Conference on Learning Theory, volume 134 of Proceedings of Machine Learning Re- search, pages 2463-2498, PMLR, 2021.

---

[20] S. Jaffard, S. Vaiter, A. Muzy, and P. Reynaud- Seynaud-Bouret. Provable local learning rule by expert aggregation for a Hawkes network. In S. Dasgupta, S. Mandt, and Y. Li, editors, Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, volume 238 of Proceedings of Machine Learning Research, pages 1837-1845. PMLR, 02-04 May 2024. URI. https://proceeddngg mlr. press/ v238/ html
[21] R. Kempter, W. Gerstner, and J. L. Van Hemmen. Hebbian learning and spiking neurons. Physical Review E, 59(4))4498-44 4514444144444 1999.
[22] R. Kempter. W. Gerstner, and J. L. V. Hemmen. Intrinsic stabilization of output rates by spike- based hebbian learning. Neural Computation, (3(12):2799-2744 2001.
[23] S. Kumar and Sarkar. Oja's algorithm for streaming sparse PCA. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors, Advances in Neural Infor- mation Processing Systems, volume 37, pages 74528-74578. Curran Associates, Inc., 2024.
[24] T. L. Lai. Stochastic approximation: invited paper. The Annals of Statistics, 31(2):391 406, 2003.
[25] N. Lynch, C. Musco, and M. Parter. Winner-Take-AN computation in spiking neural networks, 2019. URL https://arxiv org/abs/ 2207 21078.
[26] P. Mertikopoulos and W. H. Sandholm. Riemannian game dynamics. Journal of Economic T'heory, 177:315-364, 2018.
[27] P. Mertikopoulos, N. Hallak, A. Kavis, and V. Cevher. On the almost sure convergence of stochastic gradient descent in non-convex problems. In H. Larochelle, M. Ranzato, R. Had- sell, M. Balcan, and H. Lin, editors. Advances in Neural Information Processing Systems, volume 33. Curran Associates, Inc., 2020.
[28] A. Morrison, M. Diesmann, and W. Gerstner. Phenomenological models of synaptic plasticity based on spike timing. Biological Cybernetics, 98(6):459 478, 2008
[29] A. S. Nemirovsky and D. B. Yudin. Problem complexity and method efficiency in optimization. Wiley-Interscience series in discrete mathematics. Wiley, 1983.
[30] K. Neumann, C. Strub, and J. Steil. Intrinsic plasticity via natural gradient descent with appli- cation to drift compensation. Neurocomputing, 112:26-33, 2013.
[31] M. Oster and S.-C. Liu. Spiking inputs to a winner-take--a network. In Y. Weiss, B. Schölkopf, and J. Platt, editors, Advances in Neural Information Processing Systems, volume 18. MIT Press, 2005.
[32] J.-P. Plister, T. Toyoizumi, D. Barber, and W. Gerstner. Optimal Boike-Timing-oe Deeeedn- Plasticity for Precise Action Potential Firing in Supervised Learning. Neural Computation, 18 (6):1318-1448 2006.
[33] G. Raskutti and S. Mukherjee. The information geometry of mirror descent. IEEE Transactions on Information Theory, (3)):::1511145,,, 2015.
[34] H. Robbins and S. Monro. A stochastic approximation method. The Annals of Mathematical Statistics, 22(3):400-407 1951.
[35] D. Sanger. Optimal unsupervised learning in a single-layer linear feedforward neural net- work. Neural Networks, 2(6):459 473, 1989.
[36] B. Scellier and Y. Bengio. Equilibrium propagation: Bridging the gap between energy-based models and backpropagation. Frontiers in Computational Neuroscience, 11, 2017.
[37] J. Sjöström and W. Gerstner. Spike-timing dependent plasticity. Scholarpedia, 5(2): 2010.
[38] S. Song and I.. Abbott. Cortical development and remapping through spike timing-dependent plasticity. Neuron, 32(2):339-350, 2001.
12

---

[39] S. Song, K. D. Miller, and L. F. Abbott. Competitive Hebbian learning through tenke--imiig-- dependent synaptic plasticity. Nature Neuroscience, 3(9):919-926, 2000.
[40] L. Su, C.-J. Chang, and Lynch. Spike-based winner-take-all computation: Fundamental
limits and order-optimal circuits. Neural Computation, 31(12):2523-566 2019.
[41] S. C. Surace, J.-P. Pfister, W. Gerstner, and Brea. On the choice of metrie in gradient-based theories of brain function. PLOS Computational Biology, 16(4):e0007640, 2020.
[42] T. Toyoizumi, J.-P. Pfister, K. Aihara, and W. Gerstner. Generalized CeenenstockCooppeMMum rule for spiking neurons that maximizes information transmission. Proceedings of the National Academy of Sciences, (0211)))239-524
[43] T. Toyoizumi, J.-P. Pfister, K. Aihara, and W. Gerstner. Spike-timing dependent plasticity and mutual information maximization for spiking neuron model. In I.. Saul, Y. Weiss, and I.. Bottou, editors, Advances in Neural Information Processing Systems, volume 17. MIT Press, 2004.
[44] T. Toyoizumi, J.-P. Pfister, K. Aihara, and W. Gerstner. Optimality Model of Unsupervised Spike-Timing Dependent Plasticity: Synaptic Memory and Weight Distribution. Neural Com- putation, 1933)663-677, 2007.
[45] A. Vigneron and J. Martinet. A critical survey of STDP in spiking neural networks for pattern recognition. In 2020 International Joint Conference on Neural Networks (IJCNN), pages 1-9, 2020.
[46] S. Weissmann, S. Klein, W. Azizian, and L. Döring. Almost sure convergence of stochastic gradient methods under gradient domination. Transactions on Machine Learning Research, 2025.
[47] A. Wibisono, A. C. Wilson, and M. I. Jordan. A variational perspective on accelerated methods in optimization. Pruceedings of the National Academy of Sciences, 113(47), 2016.
[48] Yang Dan and Mu-ming Poo. Spike timing-dependent plasticity of neural circuits. Neuron, 44 (1:23330, 2004.
[49] A. I.. Yuille and N. M. Grzywacz. A Winner-Take-Al mechanism based presynaptic inhi- bition feedback. Neural Computation, 1(3):334-347, 1989.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Claim (connection between Hebbian STDP and noisy gradient descent) is justified in Section 2. Claim (alignment of postsynaptic neuron with input neuron of highest intensity) is proven in Theorem 2.2. Claim (connection to mirror descent) is addressed in Section 3.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: The limitations (analysing small network and weight explosion) of our anal- ysis are included in Section 5.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The au- thors should reflect on how these assumptions might be violated practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the ap- proach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might be used reliably provide closed captions for online lectures because fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations their approach to ad- dress problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
14

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: The proof of the main result (Theorem 2 outlined in Subsection 2.1. Its formal proof is carried out in the technical appendix. The remaining results are proven in the technical appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theo- rems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be comple- mented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclu- sions of the paper (regardless of whether the code and data are provided or not)?
Answer: [NA]
Justification: The results of the paper are of theoretical nature. Figures and serve only illustrations. The code used to generate the illustrations in both Figures is included in the submission and will be made publicly available.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all sub- missions to provide some reasonable avenue for reproducibility, which may depend on the nature the contribution. For example
(a) the contribution primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to re- produce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
15

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case au- thors are welcome describe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [NA]
Justification: The results of the paper are of theoretical nature. Figures and serve only as illustrations. The code used to generate the illustrations in both Figures is included in the submission and will be made publicly available.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (attps://nlps.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so No is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https:
/ //nips.com //nips.cc/pubiice.. htwmbbic//guid///ddeeeeeeee
sisssuubbbisoooooiooc..oo for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [NA]
Justification: The results of the paper are of theoretical nature. Figures and serve only illustrations. The code used generate the illustrations in both Figures is included in the submission and will be made publicly available.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question:
Does the paper report error bars suitably and correctly defined or other appropri- ate information about the statistical significance of the experiments?
Answer: [NA]
16

---

Justification:
The results of the paper are of theoretical nature. Figures and serve only as illustrations. The code used generate the illustrations in both Figures included in the submission and will be made publicly available.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
It OK to report I-sigma error bars, but one should state it. The authors should prefer- ably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errоrs is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [NA]
Justification:
The results of the paper are of theoretical nature. Figures and serve only as illustrations. The code used to generate the illustrations in both Figures is included in the submission and will be made publicly available.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't i make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: The results of the paper are of theoretical nature and conform, in every respect, with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
17

---

The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer:
The results of the paper are of theoretical nature and we are not aware of potential
societal impacts.
Justification: [NA]
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact spe- cific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitiga- tion strategies (c.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The results the paper are of theoretical nature and do not use data or models that have high-risk of misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards allow for controlled use of the model, for example by re- quiring that users adhere to usage guidelines restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
18

---

Answer: [NA]
Justification:
The results of the paper are of theoretical nature and do not use assets.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms use in the pack- age should be provided. For popular datasets, paperswi thcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the cense of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's 's creators.
New assets
Question: Are new introduced in the paper well documented and is the documenta- tion provided alongside the assets?
Answer: [NA]
Justification:
The results of the paper are of theoretical nature and we introduce no new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the pa- per include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Question:
Answer: [NA]
Justification: The results of the paper of theoretical nature and crowdsoureing exper- iments or research with human subjects has been conducted.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contri- bution of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code Ethics, workers involved in data collection. cura- tion, other labor should be paid at least the minimum wage in the country of the data collector.
Institutional review board (IRB) approvals or equivalent for research with human subjects
19

---

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The results of the paper of theoretical nature and crowdsoureing exper- iments or research with human subjects has been conducted.
Guidelines:
The answer NA means that the paper does involve crowdsoureing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equiva- lent) may be required for any human subjects research. you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: Our results were obtained without the use of LLM.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non sta-sadddard components.
Please refer to our LLM policy (https://neun) http://ppp..pccoooeeeeee///////0 for what should or should not be described.
20

---

A Technical Appendix
We first study the loss landscape
p p+ - 1=1
p
Lemma 2
identifies the stationary points if we view the landscape function on R
Proof of Lemma 2. The formula Eq. (7)) for the gradient (L(p) shows that the set of critical points
given by
Crit = (p E p: VL(p) - =0}
= {p p ps E {0, |||l))))>> vi E [d]
{D E : : 3n (l......0 d), S S C [d] with # n such that p
1 n jES
e
To identify local the extrema, we compute the Hessian matrix
((p) 2pp T + |||| diag(p) E dRd p R
where diag(p) E Rdxd the diagonal matrix with diagonal entries given by Substituting a critical point p" with [d] non-zero entries yields (up to permutations of rows and columns)
J(p') = Jn 0
Id-n)
Jn I, + 1nx E [d]
where 1 1xx is the matrix consisting of ones. The corresponding eigenvalues are
with multiplicity 1 and eigenspace E,, == span >i= e;),
< 0 with multiplicity n 1 and eigenspace E,,, with multiplicity d and eigenspace span(en en+ t.. ea);
(20)
where E, is the orthogonal complement of E,, in span(e),...,,e en). Consequently, only those critical points p Crit are local minima, which have local minima attain the same loss 11/12 and (pp) - oo as ||||| - oo, every local minimum is also global minimum.
i.e. p {ei,..., e2)- Since all
Remark A.1. The eigenvalues of the Hessian of the loss function computed in Eq. (2L) also imply that n 2, then p' Crit saddle point in Rdd Interestingly, when restricting to directions within the probability simplex, the case n d is not saddle point, but maximum, since the direction Dd e; orthogonal T.
A.1 Proofs for Subsection
In the following we will always assume that
Pi (0) 2 max p:(0) + A,
(21)
for some (0, 1). This is a deterministic constraint. The randomness occurs because of the noise in the updates. We assume that all random variables are defined on a filtered probability space (2, P),, and denote by F,, n abuse of notation we also introduce F_1 {0, R2.. In particular it then holds that p., is Fa-1- measurable for 0, 1..... The starting point for the proof of the linear convergence of STDP given by the following Lemma, which explicitly bounds the enrr term in the Taylor approximation contained in Eq. (L). Recall that [K) < Q, for all i [d], k 0,1....
1. ... the natural filtration of (B (n), Z(n))nEN. By slight
Lemma A.2. For i [d] and l 0, 1, ... define
{(k) == p.(k)) (A())(E ( (k(k) Σ (K)) FrI1 (Y((K) Σ ps (K))))()))
(22)
21

---

and assume satisfying
< 1/Q. Then for any i E [d] and k = 0, l...., there exists random variable 0 (k)),
2Q
10.(k)] S a² (1 (1-Qa) ((- (L)((()))))) pc())),
almost surely,
such that
pi(k+ + (k) - (kk).
Proof. By definition
1+a (Yk
(k + 1) = (k(
k 0.1,.....
i [d]
1 + a P))))
Now, for a, b E [-Q, the first two derivatives of the function
f(x): (0. ( RR x ++ar
1 + bx
given by
f((x) = a(1 + bz) - 6(1 + ax)
a
(1 + )x)
(1 + bx)
f"(x) -26 (1 + bz)
Thus, a Taylor expansion around 0 gives that there exists some y E (0, such that
a b
f(x) = 1 + (a b)æ
(1 + by)3
Hence we obtain that for some a0,
p:(k: 1) pı(k) + ap)(kk) (Y:(k) Σ (1))))
1 P3 (K) (k(()) (4)),,))))))))
a² (k:)
(1 + P (A)) 3
Using that [Y(()) < Q almost surely for all i E [d]. 0, 1,... the absolute value of the error term can be bounded as follows
Σ (1))(() a² p'k)(k)
(Y)()) (Y:(k)) E (b)))
<a²p((k)
(1 (1-Qa)3 (((k)) Σ ())))
Qa)3
< (k)
(I Qu13 (11 (()))(()) Y+(b)| +
Q
Σ ((()) j=1
2Q²
S sa² (1 - Qa) 3a)32% (1) - (p:())).
Since (kk) is messuuualrl for any E [d] and E[Y;(k) | = p:()),, we also obtain
(k) pc(k)) (1() 1|p(1)]1 Y((k) Σ (A)))
which concludes the proof.
22

---

For A given in Eq. (2I)), we define sequence of benign events
kkk) = p1(u) max Pr(u) + i 2....
A
Vu E [81).
k= =
2
and due to the assumption Eq. (LTD) we set 22(0) = S2 On the above events, the gradient is bounded away from zero and
Pi (k( d j=1
1
+ -p((t)))))(()))) Σ (pi(k) p,))
(d 1d
+ d'
(23)
2d
Using these properties, we can prove recursive upper bound for pı().. Proposition A.3. If
(1 - Qa) 8Q2 A.
0<as
then, on the event (2(k),
1 Pi S (0))=(110))))
(d (d-1)) (1- at)
4d
Proof. By definition, have the event 12(k),
d
Pi - 11p()) Σ ps (4) -b
j=1
- (1-
(24)
The constraint imposed on the learning rate implies that 1/Q and Lemma A. becomes appli- cable. Now, combining the previous incquality with the assumption on a and applying Lemma A. with = as well as Eq. (23),, gives, the event 12(k).
P1 (k +
$ pı (k) - a
PI
+ (k) + (A))
< (k)
a
A
(1 Qa)3 (0133))1((0 (1-)))))))) + (k)
2Q²
2
A
$ PI
api(k) (1() + acfi(k)
4
< (1
A
+
(d -
p1 -1- +
(b)).
4d
This concludes the proof.
Having understood the dynamics of p the favourable event R2k), we aim for lower bound for its probability. A key step the following Lemma, which states that (2(k) fulfilled as soon as
(k) = Σ (L) k 0
(25)
t=
with (C) defined in Eq. (22)) exhibit uniform concentration behaviour.
Lemma A.4. Define the sets
d E(K) = N Ey(k),
A1 4
E2(k) = max M2 < wE]k]
k= 0,
5=1
Then if
0 < < 21 Qa)3
16Q²
the following set inclusion holds for any k 0, 1,..
F((k) C Ω(k + 1).
23

---

Proof.
Let U E (2.....d) be arbitrary. It follows by Lemma A that on (kk) the bound
Pulk + 1) = (k) + apn (A) - [Ip({)]]²) - (E:(k) 02 (k)
S (k) + ap, (p) [ID(k)]l²) - (k) holds. Consequently, on (2())). we have (i + 1) - Pu(k + 1) = pk) - Pkk( + (k) - Pu (D)) - (ID(K)][²)
+ (I)(()))))))())))))))))))))))))) - -((5(()))E1(4)) - 01 (1)) + (k)).
We have (k()) I 1 (k) and thus, on 12(k).
2Q
(k) + (k) aa?
(pi(k)(1 - (p((k) pu (k)(1 p((k)))
(1 Qay3
2² 2Q
(1-p((k) + pu((k)
(1 - Qa)
4Q²
2
(1 11-pil (P))()))
(1 Qa)²
A2
4 (1 - P1 (k)):
a
invoking the constraint the learning rate in the last step. From the assumptions on we deduce that on the event (2k))
Pi (k + 1) - Pu(k + 1) - (x))
2 (k)) - (kk) + aalp (k) - Pu (1)) (pi (A:) - ||I((A)]3²) - a A
(1 - pi (1))
4
A
A2
2 (k) (k)) + a (p) (k) Pr. (1 - -1)) -
4 (1 pi (k))
2
A2
(1 - pi))) a " 4
42 (Pi(k)
2 (k) (k) +
4
2 (kk) (k)),
where we applied Eq. (2E)) in the third to last inequality. Because of ((k)) C ((k-1) it then follows,
(pi(k 1) Pa (k- 1)) (a()) 2 (p1(k) (x) + (K. E [[)))]a(k))
1000 (p) pa())) (K. & (A))
This gives
(pi(k + 1) - Pr + 1))
2 =aam (p) - (0) + Σ ((Ex(()) - =((0))))
(26)
t=
2 Ala - (kM](4)) - (
We want to prove by induction that EAA) C 9(k + 1) for all k = 0, I,.... For O, this directly follows from Eq. (2A), since 22(0) Ω due to assumption Eq. (2I)) Assume the assertion holds for some k- = 0, 1, ..... Hence, holds (k: + 1) S Fi(k) C (k 1), such that for any u E (2..... d}
it holds Elk + 1) by Eq. (26)
(pl (k + 2) Pu (K: + 2)) >
[M, + 1) - ( + 1)
A
> 2
which proves the assertion.
Having assembled the previous results, we are able to prove the linear convergence of STDP stated in Theorem Me As Proposition A- already suggests the desired behaviour of p Q(k), the main
24

---

part of the proof is to show that (2(k) is satisfied with large probability. For that we deploy Doob's submartingale inequality, which states that for martingale (Xn)nen: any p 2 and any 0.
max 1Ein
X1, 2 v)
(27)
WP
This will be applied to derive lower bounds for the event E(k) defined in Lemma A which are also lower bounds for the probability of ( by the same Lemma. For the reader's 's convenience we restate Theorem 2 before giving its proof.
Theorem 2.
Given e E (0. 1), assume
A
Pı (0(0) max Po(O) 0 and 0<as 16Q2 2...dd
A
(1-QQ))*
1 256(1 Pl (0))
(1 +4))))
Then there exists an event O with probability 2 1 8/2 such that
[[[[p()) - ne] s 2(1 - (1)) exp
4
for all h: = 0, 1,
16 d
Consequently, given 8 0, it holds
(()((()) - eill 2 26) Se
for all 2 <A + dA)
16d
log
4(1 P1 (0))
Proof of Theorem 2.2 The recursive definition ensures that pkk) is F measuuuul Thus, also (2k) E Fk-1 for any k 0, I..... One can check that (M)( defined Eq. (25), forms martingale for each E [d]. This allows us to apply Doob's submartingale inequality. To apply it with p 2, we deduce the following bound on the second moment,
E[M1 (
= 2
((0)))))))))))))))))(()))))))(
t=
k
(((((())
(61()))))(())))) io0=0iif
Σ
k
&
(((((())1(1))
Σ
t=
k
(((()())
(B E (000 (r(00) EE(0))(0)) 2 ]mre] d
2
E
[())(0))
t=
2
E
[
[B (1( - Σ (00) - -xx((0 Σ ) -
Lace] (M((0) Σ ()(0))())) (F= 1000 ( - Σ (00)00))F)) [nre]]
t=
2
E
5=1
2
E (x)(0) - Σ (0)) Lxxe]
-
1=1
Σ (0)) Lmmo]
E
2=0
2=
25

---

2a² Σ
18 [(((1P((0)))(0) 2
1n + (E ())())))) 2 ]
6=0
==2
=2QPa² 2
Σ
I8
(Σ pi(()) (Eprle) Iace)
1=
< 400ºa²
Σ
EB
(1 - p1 (f))
1=
where we used that E (k2) F = O. for any 122 > ki = 0,1,..., together with (k)] < Q. the inequality (a + b)² s 2(1=² + 12) for a,
and - p S 1. Arguing similarly we obtain for
(..
E (M, (A)3) a²E
(G(()) (((0))
8=
4 Σ Ee(())()) m0o) j=1
S a² Σ (P)( (y((()
t=
S 44Qºa 2
Σ E ppu00))a00
6=0
Hence, applying a union bound, Doob's submartingale inequality Eq. (27) with p = 2 gives for any k 0.1....
P(E(K)) 11 P
U max M3 A/4) j=1 [E(M]
d
2 1 P
max [E(k)
MMj(u)] 2 A/4)
3=1
a²
k
d k
2 1 64Q²
E
(1 - (e)
+
E
A
8=
j- t=
= 12-Q a²
k
E (1 - pi (0)
t=0
Proposition A. gives for any l: 0, 1, ... the bound
E[(1 - [[[1-p( + 1o(+ s [[ - p1 (A: + 1))()()))
S E (1
A
2 (d - (1) pi ((2))) + aEi (1)) 1nw)
a
4d
(1 a 4d
A
1
+
A
(d 11)) B[(1 - pı (k)) 1n(a)
which implies
B[(1 - pi (k)) [m(k]] 5 (1-) (1 (1-a
A
( + 2 (d (d- 1))*
(28)
4d
We set
n s((k) = pi (w) 2 max p;(1) + マタ Yu VUEN
:=.....d
k=
The continuity of probability measures and Lemma A.4 then imply
P(O)= lim P(((k))
lim F(E(k))
k
26

---

a²
2 1 - 128Q2
1R
(1
p1
(0)
2
(d (d-1)))
2 - (I))
4d
2
t=
da
= 1 11
- Pi()))
A
2 + A - 1))
2 1 - 2048Q (0))
a
A
+
2 1
2
where we used that we can assume d 2 2 without loss of generality. Additionally, Eq. (2S) and the elementary inequality 1 V exp( which is valid for any real number x; give
BI(1 P1 [a] v [ P1 (4)) [((k)
- (1 pi (0)) (1
A
A
(d - 1))
0:
1
+
4d
2
S P1 (0)) exp
a
A
1 1
(d - 1) 1))
4d
2
When d 1. the right hand side of this inequality is 0. For d
2
2, we can also use the bound
2 d/2. Together with
(P(k) e1ll1 (k()) Σ (k(k ((1)))
this concludes the proof of the first statement. For the proof of the second statement, we apply Markov's inequality to obtain
P ((lD(()) - eill1 2 26)
$ P((C) + + - -eil110 26)
$ 8
a {4- + + 8-1
2(1 pt (0)) cxp
16 -
2
Hence, if
-1
a
4 A
+ 4²)
4(1 p1 (0))
1: Σ
log
16
d
Eō
16d
4(1 - Pi (0))
log
aA(4 + dA)
8ō
then,
((lp(()- erlli 20) SE.
Lemma A.S. Given an initialization of the weights w(0) = (w) (O).... ...., wa(O)) consider two d- dimensional intensity vectors X (Ai,..., Xa) _ (~1,....,, da) with positive entries. Assume that A wi (0) > max;=2.....d Xaw, (0) and (awa(() > max::=1....d d-1 (ow; Assume we run the learning rule Eq. (E) with intensity X until time K* and then, for l > K change the intensity to X and run the learning dynamic until time + oc. If K* is small, in particular, if K* = O, the above convergence result can be applied to show that the dynamic converges to the corner ea- However, for any E (0, 1) and all sufficiently large K (depending on E), the dynamics will converge to e with probability 22.
27

---

This result shows that the dynamic can be primed at the beginning to end up in one regime. Despite the noise and the infinite amount of data, the dynamic is unable to escape this domain of attraction. From the proof, one can derive quantitative bounds for K
Proof. Given e E (0, 1), choose E (0, 1) such that
8
< 1.
(29)
1F1
A 8)
Let A = (0()) - maxi Pi) Given E, choose
(1(1-p((0))
16d
(( p1 (0))) (0)))
K
log
ad(4 (4 - dA)
Eg
By Theorem 2. this guarantees that
P(Ip(K*)- - e1l11 2 20) S 8.
On the event [lp(K*) - e1 1a < 6, we have
X (K*)
Xpw; (x (K*)
X w((+) < 6, and max
1#1 A w(K') < 8,
which can be combined into
8 max Aaw; (K*) < 8x w(K") <8TT < XIw1(K))
cifll
Using the inequality Eq.
we obtain
max (Kx < (1 8) 8
2 1 X1 (K*) (K*)
(
if
This means that restarting the learning rule Eq.
(G) at time K with intensities Ar...., Aa and max Pi(K") Applying Theorem 2.2 again
weights w/ (K*).... wa wa(K"), shows that pı (K*)
shows convergence to e with probability 2e.
A.2 Proofs for Subsection 2
This section contains additional material on the gradient flow Eq. (H), as well the proofs for Lemma A.A and Theorem 2
For d = 2, the gradient flow admits explicit solution. In this case, p(t) (P)(t), PT If p(0) = (1/2, 1/2) then this is a stationary solution and p(() p(0) (1/2, 1/2) for all / 2 0. If pi 1/2, then,
(t)
2VCe-++1 with C
1.
(30)
(2p (2pi 1)2
If Pı (0) < 1/2, then (t) = 1- (t() > 1/2 follows the dynamic in Eq. 50 immediately implies that (t) converges exponentially fast 1.
This formula
Proof of Formula Eq. (BA). Throughout the proof we set p(t) == P1 (t) and do not use the previous notation P1 P2()))(t) for the first and second probability. For 2, the gradient flow ODE Eq. (9) becomes
d के p(( = M()2 - (p()2 + (1 - p(())))))))))))))))))}}))))))p = =3p(1)² 2p(1)3 - p(t)-
(31)
Rewriting this in the variable u(t) = 1 - 2p(t) gives the dynamic.
u(t) di p(t) -6p(t)² 4p(t)3 + 2p(t) 1 _ 1i-ully (l-u(t)))u(()
d
28

---

This is solved by u(t) 1/ VCe since
dl VCe++I - (-
Ce (Ce 2 + 1)3/2 w(t)))))
Thus, p(t) = {1 ut)) solves Eq. (I... Finally, C' is determined by the initial condition p(0) 31 (1-1//CC1]
The following lemma summarises different properties of the gradient flow Eq. (9). In its statement, differentiable on [0, 1] means differentiable on (0, 1) and continuous on [0, 1].
Lemma A.6. The gradient flow Eq. (9) exhibits the following properties.
(a) of [0, 1] + I. is a convex and differentiable function, then t + Ei= ((p((t) is monotone increasing.
(b) Let i,j [d] with + j. If (p) > Pj P5(0); respectively p:(0) = P;)0), then pi(l) > PT((), respectively pı(1) = ps(1), for all 2 Moreover, if
A (0) max pi(0) > 0,
22.....
then
Pi pi( 2 max Pr(() + A for all 12 0.
=
Lemma implics that the g-norm t + pg ly is monotonically increasing whenever 1 S < co. The result also implies that if instead / is concave and differentiable, then 1 + Σ ((p((1) is monotonically decreasing.
Proof of Lemma
(a) Since is convex, o' is monotonically increasing. (T),..., qa), we have
Thus, for a probability vector q
( (q: - [| =
(a)) 96
Σ Σ
93
9
j=1
j=1
Σ
g - 9.)
Σ Isiidd
( - o' o'(9;)) -9,)
0.
(If ò is strictly convex, then strict equality holds if and only if q is one of the stationary points described above.) Using this and the gradient flow formula
d Σ (l)((1)
Σ
d'vi(())
d p0 Σ ((0)(0((0))) -Ip((1))) 2
proving the result.
(b) By definition it holds for i, [d]
d (p((1-); - (() = (i()) - -,,0)))),(000000) -1b(0)))
de
- ((x(00--:1))))00+=+(
+ (lP()))))
(32)
From this we deduce that
(t))) (t()) = (p:(0) (0)) exp (f. (i(() + ps(ss) - 1Ip())
29

---

which concludes the proof of the first statement since the exponential function is strictly positive. To prove the second statement, have
pi(t) = pi(0)) + S ((o((p(()-I(()))))) ds 2 PI(O) t,
and similarly, for any i E (,...., .d.
(t( S p:(0) + + + A.
Hence, whenever E [0, A/2], we find
Pi(1) 2 max pi(t); 2
which also implies
PI(1) 2 2)))) max pz(() p()) |I(())
i=.....
for all & [0, A/2]. Therefore for any E 0, 0,2/2,, E [d] it holds
d dt (p)(1) p()) (pz(1) p.())) (px (t) p.(() - [Ip(t) 2 0,
which implies for any i E ( d) and E 2/2 (t) (t) 2 Pi (0) (0) 2 4,
Applying this argument iteratively concludes the proof.
For the reader's convenience restate Theorem 2.4 before giving its proof.
Theorem L
Assume
(0) 2 max Po(0) + +
22....,
for some A > 0. Then
-(t]li s 2(1 - (0)) exp (- d (1 (d (d-1) le
that is linear convergence of pt) + e1 as t - co.
Proof of Theorem 2 Arguing as in Eq. (a and Eq.
Lemma A.6 Eq. (6) implies
(1 - pt))) (p)) [Ip(t)]
dt
5-
where
= d ((d - 1)4 1
Grönwall's inequality entails
1 1- (t)) S (1 p. (0)) exp(-
which gives
[P(() eilli 1
Σ pi(1) 2(1 (2) < 2(1 (O)) (xpeexxp(-i0
30

---

A.3 Proofs for Subsection 2K
In this subsection, heuristically derive the expression for the probabilities
Ayw (4x)
j == ;=1....,dd
(33)
Σe=1 \ewe(ts)
in Eq. (12). To this end, we assume that the weights are small compared to the threshold S, that the weights are only updated at the postsynaptic spike times, and that Ei= Aewa(t+:) > S. For convenience, we write Wy for wr(4k) and all é E [d]. The constraint Di=1 Apw, > guarantees that after the postsynaptic spike time tg, the membrane potential Y, will again reach S and thus emit another spike time 1++1-
Taking the expectation of the membrane potential Y, = -E EEETn(Iu wye (1-T) with respect
to all except the jth spike-train, gives
wede I' c-((-o) ds
Ze:= Σ
wye w (1-7) Σ
TETYN(tA.))
Σ
Wye (1-T) +
Wie
Σ
wrde(1 - (e--tti)))
TETyn(te.
(+)
for all t. < < t-+1-
Introduce t" = in (1 2 to: Ze 2 S - -w²) and write + for the first time after t where
d
Z. + (f--tt
Σ we wede(1 (tc
reaches the threshold S. If there sufficiently many neurons, the probability that the jth presy- naptic neuron spikes time is small and will be neglected. We have Ze. = - Wy such that
Vt+ Wy. Approximating - x +
t' gives
+ f"
W
Ees wede
The jth presynaptic neuron causes the next postsynaptic spike if and only if it spikes in the interval (t", t.). The spike times of the j" presynaptic neuron are generated from a Poisson process with intensity Ay. Thus, if U ~ Poisson(\),( - f*)), the probability that the jth presynaptic neuron spikes (t", (t) is given by
Wids
P(U + 0) = 1- PU = =1 exp (- Ay - ()) = 21(t -() Eess wede
We can moreover approximate the denominator on the right hand side by the full sum Ed we Since the probabilities add to one, we must have et" z 1. This shows that the probability of the 1th presynaptic neuron triggering the first postsynaptic spike after lg is approximately given by Eq.
Lemma A.7. Consider the setting outlined in Subsection 2.3 the same, then the probability that the jth neuron triggers the next postsynaptic spike after t is given by
If some time point t >>>>> O, all weights
EL X
Proof. Since all weights are the same, we can denote their value by The 7th neuron causes postsynaptic spike if and only if it is the first one to spike after the postsynaptic membrane potential Y, has reached a level > S w. As t" inf > t: Y, S - w/ is jump time and stopping
31

---

time, we can restart the process at t" As the increments of Poisson processes are independent, and the time between the jumps is exponentially distributed with parameters >y. the probability that the 5th neuron causes the next presynaptic spike is given by
P P(X, min(X
where (Xi) are independent random variables satisfying X, Exp(A,). U Exp(A) and V ~ Exp(X) are independent, then, U A Exp(A + X) and P(U S V) X/(2 + X). Thus, min X; Exp(Eits As), and
P(X; = min( X1 Xa)) = P (X3 S min
X Σ de
[F)
The above lemma and the previous discussion give motivation for the form of the probabilities in settings, where the weights are small compared to the threshold S or equal. We now give another heuristic, motivating our modelling choice. Let Y be the postsynaptic membrane potential. Then, in expectation Y grows linearly with slope A w. Furthermore, input causes postsynaptic spike if, and only if, spikes at a time at which Y 2 S - Wi, where > 0 is the threshold level. As Y's wyy holds is approximately
growth is approximately linear, the amount of time in which Y 2 equal wt/ / w. Now the probability that input jumps in interval of length wi/ X given by e-exp(-a0 exp(->xu,/> w) > w, which is exactly our modelling choice in the independent setting.
A.4 On the connection to entropic mirror descent
An alternative approach to connecting our proposed learning rule Eq. (E) for the probabilities p and the entropic mirror descent discrete-time is follows. The entropic mirror descent step Eq. (IE) with Kullback-Leibe divergence and potential can be solved explicitly and yields
Pe (c(k) (xp(ex((//(((()))))
Pk(k +
i [d] 0, 1.....
(34)
( (x)[<()exxp((((((()))))))
see Section Beck and Teboulle [5]] for details. With f(p) = L(p) = ||212//2 and the first order approximation exp(x) s + x for small æ deduce
pilk) (x())ex((=((V((p())
pilk) exp(()x(k()))
pe(k 1)
Σ 1 Py (x)))))x(-u((((((())))
Pr (k) + (p)(k))
プØ= Dy=1py (k) ap,
d
for any = 1,...,,, . and k 0, 1,...... As our proposed learning rule Eq. (E) is a noisy version of the last line, it naturally connected noisy entropic gradient descent.
A.5 Theoretical results for the alignment of multiple read-out neurons
For weight vector w E Rd with nonnegative entries let it argmax and define the cosine-projection
e;
Pw ||w|e
Assume that A > > A and consider first learning the first weight vector W1 using the learning
rule in Eq. ( while the remaining weight vectors W2,..., , Wa fixed. Theorem DM implies that after K iterations, we have W, so Cj. By projecting onto w = Pwi(K) and setting wz( <- (0) wi (0) ww// [will we ensure that [p2(0)]2 max (oax:=1...((((((0)) and Theorem applies and yields Pa (K) e2. Proceeding successively, we arrive at Algorithm Z
32

---

Theorem A.8. Consider Algorithm 2 Assume that X > ... Aa and the minimal gap A
1 max (0],) 0 positive. Then we have
P(P + +0
K + More precisely, let k/(1 + k) for k
X1/ max;=l,....d de and
Then
16d
P(P"=11 (1 8)a for K aA(4+ dAj
log
Ed
Algorithm 2: Sequential alignment of multiple output neurons
Input: K N: number of iterations for each learning period, ((0) E Rddd, weight initialisation.
for == -,..... do
2.
j 2 2 then
3
(0 t w, (0) -1
w W
end
for k == 1,,,...,, do
6
Receive By(k) ~ M(1, (k)) with (k) + A o (k)/A/A wy(k) and
(k) ~ Unif([-1,1]1 1] from spike trains;
Update
wy(k 1) + aw, (k) 2 (B) + 2,(}));
Set w = (K) to obtain pj = A wj.
end end
Output:
The weight evolution ((k) = [w, (ks) ... W. wa(k)] k = O.... K, probability
evolution (kk [pi (k) (k)), .,..., K and projections
p = p Pal-
Before proving Theorem A.N. start with auxiliary result on the order of the weights when the probability vector is close to standard unit vector.
Lemma A.9. Assume that 0 < Amin = min;=1....d di S Amax = max;=1....dd d < Do and Amax- Consider a weight vector w E Id with corresponding probability vector p w A/w X and let 8 Then the condition Pr 8 implies that max;=2.....d Wi S
let K Amin/ Amin//uax-
Wik 8 5).
8 Proof. Since 1 - pt S 8 we know that Σde Ei=2 dww; S I-8 Nrw, By bounding the X; from above and below we find
Amin
W
Amax W1,
such that
8
max
wi <
K -ww
2 2.....
Proof of Theorem A.8. By Theorem 2 know that P(Ilp (K) eill 8) > 1 - for
our choice of K. Note that the projection picks out the direction argmax WI (K) e
argmax [w] (K)]:- Consequently, L. and 5 < k/(1 - x) imply that w (K) projects in the direction of with probability at least 1 adjustment of the weight vector w2(0) we remove first component, such that [2((0) = 0 and [p)( (pxx=
We find that P(p * e1) < E. By the
By the assumption on [P2(0)]2 (o.xx),....((((()) > A 0
we find that (P[[|p2 (K) e2 S 8) 2 E. By iteration and independence the training windows conclude the proof.
33

---

A.6
Additional material for the extension to time-inhomogeneous intensities
Derivation of Eq. (II): Eq. (LZ2) yields the deterministic update scheme
p + 1) ((k) @ [ + (1+a(p(A)) a(((k) p(k) p({)1)) + -(a7).
(35)
Let >> 0 be the time horizon and assume that the time-inhomogeneous intensities Aa change on the
same scale the weights and are given by X, (t) = (t E [0. T/al, where
( 77) is
universal, twice differentiable function with bounded derivatives and X(1) 2 Amin > 0, E [0, ,
componentwise. For vectors w, X, X that are of the same length, and A
X X, we have
O W
A 3
X N
O W
A
A w A + X
A +
X
A
T
Applying this with X Xa (k + A= A (k), w w(k), yields
p(k) Ao (ks + 1) w(k)
da + 1)T ((k)
AAa (k) ② w(k)
AA, (k) T w(k)
=P(k) + AAa (k) w (k) Xa (k) w(:) p())
AA, (A:)
w(k) + Aa (k) w(k)
Our assumptions on X, imply that (kk) - pkk) 2 [ 2 a for all k = 0, I..... [TA.....T/]]] This
gives p(k) @ p(k) (kk pkk) = pkk) @ pkk) (k) p(k) + O(a). In combination
with Eq. (35)) we find
p(k + 1) plk)
(ka a
w(k)
a
AAa (k) ((k) + Ao (1))TT (((k
((k)
AAa(k) w(k)
AA (k) w(k) + (k)
a
+ ((k) O p(k) (K())(()) p(k)
(k)
w(k)
O(a) + (k) [[(k
(kk)
(K) w(k)
(36)
O(a) Aa (k) w(k) a
+ +k(k) O ppk)) -P())p(k) p(k) O(a).
Sending a + O and using that AA, (1/a)//a + A((1) as a + 0 we recognize Eq. (G6) as an Euler-type scheme for the ODE
d dt P(t) di [A(t) O w(t) A (t)w(t)
p()) [((T))] w(t)
A (t)w((t + +p(t) o (p(1) 1l)
log (A(t)) p ((t) d dt
log (A(0))1 + ptt)) @ (p(t) - 11
= =t() o dt
log (A(t) p(t) - p(t) (d dt
log (A(t) + p(t) ) EE (0,
pt) O
dt
where the logarithm is taken componentwise.
A.7 Analysis of the correlated model
In the following we always assume that the off off-diagonal entries of T are strictly smaller than 1, that is, Ta, 1 for all i = =......... d. This assumption is reasonable since perfect correlation between input and input corresponds to one single input neuron with Poisson process intensity A; + >ÿ. Under this assumption it is easy to see that the quadratic form associated to maximised on the probability simplex by the basis vectors . ey. Thus, the natural question to investigate is the same as in the original model: To which basis vector does the model converge? In the following section we investigate this question and show results for the weakly dependent case.
34

---

P00-1227.418--0
-013
a00 (02-0.2344.33
201--15432223333 2711-13--22. SEGA)
Figure 6: Correlated inputs with
1
1/2
0
1
3/4
1/10
1/10
r =
1/2 1
1/2
(top). r
3/4
1
(middle),
1/10
1
0
(bottom).
0
1/2
0
0
1/10
0
Contour plot of the Shahshahani loss function I(p) - Tp on the probability simplex T
for d 3 with different overlays. Left: Three sample trajectories of Eq. (E) with different initial
configurations p(0). Middle: Stream plot of the gradient field given by Eq. ( Right: 100 sample
trajectories of Eq. (B) with p(0) (0.3, 0.3, 0.4) All trajectories are simulated with 2000 iteration steps, learning rate 0.01 and (kk) Uni 1ja).
The analysis the correlated version of the model follows the same steps the independent case. For this we assume that all random variables are defined on a filtered probability space (Ω, F, IP) and denote by F,,
2...... the natural filtration of (B (n). C(n)) NEN, and set F_1 {() Ω). The proof follows exactly the same steps as the proof of Theorem 2,2 We start with the following result, which bounds the error of the Taylor approximation of the random dynamies.
Lemma A.10. For E [d] and = 0, 1,... define
(ikk) = (.))) (E
Σ #=1
(4)) (Y((k)
Σ (())(()(())) 1=1
(37)
and assume < 1/Q. Then for any i E [d] and k = 0, 1.... there exists random variable 0, (k),), satisfying
2Q²
10:(k) < 2
(1 Qa) ((t)) -
almost surely,
such that
(akk + 11 (kk)
(p())) TP(k)))Tp(k)) (k:(k)) - (k))).
35

---

Proof. As the dynamic of p still follows Eq. (2) as in the uncorrelated model, we can argue as in the proof of Lemma A.2 to obtain that for some y E (0.1)
P: + +) pk(k) + (ps(k) (Y((K) Σ (s()))((K))
-a²p,(k) P3 (V)())))))))))))))))))))))))((
(2(()) (B)
(1 + (X))
Then, since [Y([] 2 we obtain the same bound on the error term as in the proof of Lemma
- 1 P,(K)), (1)
(((k) ( P)(K))
a (kk)
(1+1E)119)))
I a 2Q
Qa)3 P (k)((1 pe())))
Since the (k) are FR-1 meeasubbeee and C(k) is independent of Fk-1, we finally obtain [[ - p(k).
In the following we always assume for Ap, Ar > 0
(0) 2 p.(0) Ap. Vi = 2...., d,
(Tp(0)) (Tp(0)); Dr, Vi 2, .d. d,
(38)
4 v/ 11 ApAr)
0,
where,
max Taj- *jEE|
4
These assumptions for example fulfilled in the following
70.8
1
0.1
0.1
p(0) =
0.1
r
0.1
0.1
0.1
1
since we can choose Ap 0.7, Ar 0.64, v 0.1, and hence C. 8 10-4 In the indepen-
dent case, which corresponds to 0, the assumptions given in Eq. (EX) reduce to the original assumption given in Eq. (2I)). For 1,2,..., we define a sequence of benign events
((k) = (pi(w)= 2
max (u( Ag (Ip(u)), 2
(Tp(w)); + 2 Yu [k]].
Ar
max 2
2
2
Due to assumption Eq. (88), (2)0) Ω. Additionally, since the assumptions in Eq. (B imply the gradient to be bounded away from 0, we can prove the following recursive upper bound for 1 1-ppt (k). Proposition A.11. If
-1-Q0 Qa)3
0 0<aS
Ar.
8Q2
then, on the event Q(k),
1 1 Pi + 1) (015((10)
Ar (1
Ap (d - (1-((()) - oEi(k).
4d
Proof. By definition, have the event 1(k))
(Tp(k)) - ppk) TP(() =
Σ py (T)))(
j=1
Ar
>
(1 (1:))
(39)
2
36

---

The assumption on a implies a < 1/Q and thus Lemma A 0 becomes applicable. Now applying Lemma A.I with gives on the event ((k).
1 pı (k: 1)
Ar
$ - Pı (k)
a
Pı (1))((1 pp((k) + (f;(k)+ + (k))
2
År
2Q
$ - (B)
=(1 (1-Qa)² pr (1:) (T))) + (Ei(k)
Ar
$ - (k)
a
(1())( - -pi(k)) + ok:(k)
4
< (1 a 4d
Ar
Ap
(d - 1111)) (1-(()) + og((k)
2
This concludes the proof.
As in the uncorrelated setting our goal is now derive a lower bound for the probability of the favourable event (2))). For this we again follow the same strategy and rely on uniform concentration inequalities for martingales. In order apply those, require the following lemma, which states that ((k) is fulfilled as soon as
Mk(k) = Σ (0) k = 1,., E=
(40)
with (E) defined in Eq. (22), exhibits uniform concentration behaviour. In the following denote by || the row-sum norm of I, i.c. | = max,e14 Σj= Tt.-
Lemma A.12. Define the sets
d Ar ). E(k) = 0 (k)), k= 0, 3=1
(k); = { max [M](u)] - 4
1 (4, A
welk
|
Then if
0<a (1 - Qa)3
Ca.
4Q²
the following set inclusion holds for any k = 0, 0.1....
E(k) S Q(k + 1).
Proof. Let u E (2....,d) d) be arbitrary. By Lemma A. on the event (2k))
(I( + 1) Pu(k + 1) 2 pı (k) P.. (k) + a (pp Pa (k()) ((Tp(k)) -Pk) T Tp(k))
+ a (E, (A)) (1(k) + (k).
Since (k( < 1 - P1 (k) also obtain on s(k).
(k) + (k) aa² 2Q
(1-Qa) PI (p((k) (P)((())) + (p)(((()p(((k)))
22 4Q
(1 Qay3 (1- (1()).
The assumptions on a then imply that the event ((k).
Pi (A: +1 P.(k + 1) (E, (A))
2 (kk) (k) (p) (k)) (Tp())); p( - o² 4Q2
(1 (1-Qa)3 (1 p1 p1()))
Qa)3
2 PP(( (k)).
37

---

Because of (2)k) S Q2k - 1) it then follows,
(pi(k+1) Pa(k + Ta()) 2 (pi(k) - Pa (k))) ((k) + (k)(())) (i(k))
12(0) (p2 (k)))aa())))) a(5 - ((4)))
This gives
(pi (k 1) Pu 1(())
2 1n(k) (1)(0) - (0) Σ - (1
(41)
20
2 A [(()) - [M, (
Now again let u E (2, d) be given. Then it holds, on (2(k)
(Tp(k 1))1 (Tp(k 1))x
=Tp((k)) - (Tp())) + Σ - p(k)[p)
d
Σ
r (k) p(()))))))pp)))p)>>>>))))))))))) - (Ti, - (4))
j=1
|=1
2 (Tp(k))
- ((L--))((
- p(})pp)
d
+
Σ
(T1 - (t,,))) -p())pp)
j=2,j
E (Tis (aE) +0))
j=1
A, (1 - ()))) av(1 - (b))
2 (Tp(k)) - (Tp(k)) + a(1 TI,-)
4
Σ (Ti. - - (a.) +k)())
2 (Tp(k)) (Tp())), + a (11 v)
A,Ar
(1 - Qa) (1 pk))
4Q²
v a
4
Σ (1.. (k))
1=1
2 (Tp(k)) - (Tp())),
Σ (1. (k).
j=1
Hence, arguing as the derivation of Eq. (MIl) gives
(Tp(k + 1)); - (Tp(k + 1)),, 12())
2 Laga) (TP(0)) - (Tp())). ΣΣ
(1y - Fuu)
E=0j=1
(42)
d
La(a (Tp(0)) - (Tp(0)) Σ
(TIY- - r Tuy)
j=1
La(N) (Tp(0)); (Tp(0)) || max M2,(k))
sE[d]
With the above results we can now begin proving that F(k) C Ω(1 1) for all 0, 1.... We
do this by induction. For & = 0. this directly follows from Eq. (Il) and Eq. (022), since 2(00) Ω
38

---

by assumption. Now, assume the assertion holds for some ho = 0, l..... This implies E(k + 1) S
E(k) 52(k + 1). such that for any (2.....d) dd it holds on E(k + 1) by Eq. (HI)
(pi(k: + 2) (k + 2)) A - (k + 1)1 - Mi (1+ + 1)]
4
2
and additionally by Eq. (22) holds on Ek + 1)
(Tp(k + 2))1 (Tp(k + 2)). 2 (Tp())) (Tp(0)), 2||T|| max MM + 1)1
sE(d
Ar
2
2
which proves the assertion.
With the above results we are now able to prove and state the main theorem for the correlated case.
Theorem A.13. Given & E (0, 1), assume
A = pı (1))) max Pi(0) O, Ar = (Tp(O)
max (Tp(0)) > 0, 2=
i=
C = 4,Ar v 1+
4pDr)
> 0, where,
v max Tú, ifj
4
4
and
ocas 1 (11
(a-Qa)²cc A
1
Ap A
4 Dr
))
4Q²
1024(1 p1 (0))
|
d
00
Then there exists an event with probability 2 1 - E/2 such that
Dr
+ 10)16)
E [[(]p(k) - eillile] s 2(1 Pı (1)) exp
for all 0, 1....
16
Consequently, given 8 > 0, it holds
((ID(()) e1 l 2 6) S for all k 2 aAr(4 + dA,)
16d
log 4(1 (())))
Eô
Before giving the proof of the above theorem, we want to remark that Theorem A exactly recovers the result of Theorem Me in the independent case. Indeed, in the independent case equal the identity matrix and thus Ap = Ar, V 0 and c. = 42/4 hold true, which gives the result of Theorem 2
Proof of Theorem A R As in the uncorrelated case, the recursive definition ensures that p(k) is FR- measurable. Thus, also 12(k) F for any 0, I.... Then (M)(+)) defined in Eq. (HO), is martingale for each i [d]. This allows us to apply Doob's submartingale inequality. For this, we deduce the following bound on the second moment,
E[M (
a
(((E(
0=
a Ex(00x(0) - (x((0) -(x()) - Σ ()(()(()) 2 LLac]] j=1
E (())
t=
E (0) Σ (1(001(0)F==)] (xi(0) Σ ())))(())) Fe-1 ( |
EE
5=1
(x)()) - Σ ps (0))())) La0
<
0=
39

---

Σ (1-P((0))1()) - Σ (())))(())) Lace]]
a²
2=
T=2
=2Q 2
Σ E [
p))( 2 Lace]
S <4Qºa a²
Σ
[( (1))
E pı
1=
where we argued similarly for the independent case. Furthermore, we obtain for (2..... d).
[((M,(
2E
((1(0)))))))))
E=
a² Σ
B (P)()) (Yu(t)
E))())(0))
Incs)
8=
4Q0aa
Σ
E pa (P)
t=0
Hence, applying a union bound, Doob's 's submartingale inequality Eq. (27)) k 0,1....
with p = 2 gives for any
P(E(K)) = 1 IP
U max uElk
MM,(w) (4, A A
Ar
|
j=1
d
1 (4p ^
Ar
2 1
IP
max kk
My (2))
7
|
j=1
2 1 - 64Q 2 (Ap A
Ar
Σ
E
[11 -p((
+ Σ E
E ((0mm0))))))
|
t=
f=
k
Ar
1 10-122200 a² Ap ^
E (1 (1- pz (e))
|
t=
Proposition A.
gives for any A: = 0, 1,... the bound
(1 pı (k: + 1)) 12(k+1))
E (1 pi (A: + 1)) l2()))
E
(1- a 4d
Ar
[ +
Ap (d (d 1)) pı (1()) Io(k) + a& (k)
2
Dr
[1 1
4 (d - 11) ) [[1
pi (1:) La(n)
a
4d
2
which implies
(1 - pi(k)) [2ck]
S (1 - pi (0)) (1 (1- a 4d
Dr (1+ 4 (d-
(43)
We set
9(k).
The continuity of probability measures and Lemma A. 12 then imply
IP())
lim P(((k)
k0+00
40

---

lim P(E(k))
1-000
21 - 128 (4p A
Ar
-2
Σ
B
(1
pı
(1)) m())
8=0
Ar
-2
Ar
1+ 4 (d-
2 1 - 128Q2222222000 21 (1- (0))
A A
|||l
4d
8=0
1024Q a(1 (1 (0))
År
-2
d
|||
Ar + Ap(d-1))
1 - 2048Q a(1 P1 (0)) (4))
Ar
-2
|
Ar Ar(4/d+4)
E/2,
where we used that we can assume d 2 2 without loss of generality. Additionally, Eq. (233 and the elementary inequality 1 S exp(-2) which is valid for any real number æ, give
B[(1 - pı (1)) s (1 pi (k)) 1 (ocO)
(1 pi (0)) (1 11-a
Ar
1
A (d
- 11))
4d
2
(1 pr (0)) exp (-
Ar
1
Ap (d 1) kt)
4d
2
When d 1, the right hand side of this inequality is 0. For d 2 2, we also use the bound - 1 2 d/2. Together with
pp(k) eilli 11-Pi + Σ P.(k) = - (k)))
2=2
this concludes the proof of the first statement. For the proof of the second statement, we apply Markov's 's inequality obtain
(ID((k)- - eill 2 2)
S SP((())) (((p(()) - 2 26)
11
- (-a Ar
$ 2
+ 2(1 pi (0)) exp
16 d + +,, 8-1
(1
Hence, if
aDr
d + 4,))
4
4(1 4(1 (11-pt(00 P1 (0))
k
log
16
Eō
16d
4(1 (1 ((-))(()))
log
aAr(4+ + d
Ed
then,
(()p(k)-ee)) eilli 26) Se.
41


Reviews:
[{"review_id": "sVVnHlwkDL", "reviewer": "Reviewer_9Yjy", "summary": "The authors provide a new theoretical analysis of the convergence behavior of Hebbian STDP rules. First, they show that a Hebbian STDP rule can be reformulated as gradient descent on a loss function defined over the presynaptic neurons spike-triggering probability simplex. Then, they prove that such spike-triggering probability converges to the basis vector of the probability simplex pointing to the presynaptic neuron with highest initial spike-triggering probability. Lastly, they discuss connections to noisy mirror descent.", "questions": "Currently, the argument in Section 5 of the paper is not rigorous enough. Can you prove convergence of p(k) for an arbitrary number of readout neurons?\n\nCan you provide an intuitive interpretation for the loss function in Eq. 5?\n\nWhich is the practical significance of Theorem 2.2 for training SNNs with STDP rules?\n\nWhich known STDP rules fall under the formulation of Eq. (1), and which do not?", "limitations": "Currently, it is not clear which STDP rules belong to the Eq. (1) family. The authors should specify that Theorem 2.2 is limited to STDP rules belonging to the family in Eq. (1).", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength: The mathematical derivations are theoretically sound. The rigorous convergence analysis of p(k) in STDP is a potentially important theoretical result. Viewing STDP as gradient descent on the presynaptic neurons' spike-triggering probability simplex is, to the best of my knowledge, a novel and compelling idea that may inspire further theoretical developments.\n\nLimitations:\nDespite its mathematical rigor, the manuscript lacks clarity in key areas, and in its current form, the novelty does not appear sufficient to meet the standards expected at NeurIPS.\nFirst, it is unclear whether the authors introduce a new Hebbian STDP rule or analyze a family of existing ones. Lines 12–13 state they “provide an analysis of a STDP rule,” while line 83 says they “introduce a STDP rule,” suggesting novelty without clarification. Yet, Section 2.3 presents a biologically plausible STDP rule that has already been studied in the literature (e.g., Ref. 11), and that can be reformulated to match Eq. (1). This suggests that Eq. (1) generalizes a known class of STDP rules, with Eq. (11) as a specific instantiation. However, the manuscript does not clearly state which rules fall under this general form.\nThe paper's structure further obscures this point: Eq. (1) is introduced early, without sufficient context, and its connection to established STDP formulations only emerges later, in Section 2.3. This non-linear narrative makes it difficult to follow the paper’s logic.\nThe authors should more precisely specify the generality of Eq. (1) in relation to known STDP variants that go beyond the brief description in lines 27–28. Experimental studies support the existence of multiple types of STDP rules, varying across brain regions and cell types (Fino et al. J Physiol 2008, doi: 10.1113/jphysiol.2007.144501; Perez et al. Cell Rep 2022, doi: 10.1016/j.celrep.2022.110521). While this diversity is not yet fully leveraged in SNNs, the authors should acknowledge it and specify the generality of Eq. (1). \nIt is also unclear what results could be derived using the previous methods mentioned in lines 35–37, and which results uniquely stem from the gradient-descent interpretation introduced here.\nWhile the derivations appear theoretically sound, their novelty and significance remain uncertain. As the authors also point out, convergence to a winner-take-all regime has been previously observed for Hebbian STDP models. The authors do not sufficiently discuss the added value of their convergence result, which appears to be the main theoretical contribution of the paper. The connection to noisy mirror descent is also only briefly mentioned and not clearly motivated or explored. \nOverall, proving the convergence of a specific family of STDP rules to an already known regime – within the simplified setting of a single-readout neuron - constitutes, in my view, a limited theoretical contribution that does not meet the standard expected at NeurIPS. Rigorously extending their theoretical result to the scenario of an arbitrary number of readout neurons would significantly strengthen the contribution.\nMinor comments:\n•\tEq. (1): The quantity Z(k) should be defined immediately after the equation. Also, explicitly stating that k indexes postsynaptic spikes would aid clarity.\n•\tThe writing style should be improved throughout. For instance, the sentence “We continue by relating the learning rule (3)” (line 101) appears before Eq. (3) is introduced and is difficult to follow.\n•\tTo avoid confusion, I recommend consistently prefixing equation references with “Eq.” to distinguish them from citation numbers.", "was_revised": true, "final_justification": "I appreciated the authors detailed response. The additional emphasis on the novel contributions, along with the new analytical results on convergence with multiple output neurons, improved my understanding and appreciation of the work. As a result, I have updated my score to 4.", "rebuttal": "Thank you for your careful reading of the manuscript and your suggestions. The feedback did help us to improve the article considerably.\n\n- **On the class of considered STDP rules:** We excuse for any ambiguity that might have occurred during reading. As mentioned by you, we start with a widely considered pair-based STDP rule that is stated in the standard reference [11]. We now make one modification. In the original STDP rule, the postsynaptic spike timings are determined by the moments when the postsynaptic membrane potential exceeds the threshold. According to this rule, the postsynaptic spikes can only happen at moments when one of the presynaptic neurons spike. While the formula for the postsynaptic spike timings is mathematically easy to formulate, a direct analysis seems extremely challenging and has to be replaced by a surrogate. To mimic the distribution of the original spike timings, we instead consider a two-stage rule that first selects a presynaptic spike train and then, in a second step, chooses the next postsynaptic spike time among the presynaptic spike times of the selected presynaptic spike train. We put considerable effort to show that the distributions of both rules are close, see Section A.3 for details. In particular, in the setting of Lemma A.5 they are even the same. For more details, please see the first response to Reviewer jhz1. This leads then to the learning rule (3). In the revised version, we will make this point more clear.\n\n    We want to emphasise that for a rigorous mathematical treatment of the learning rules, one needs to consider a tractable model. \n\n    Thank you for pointing us to (Fino et al. 2008 and Perez et al. 2022). In the revised version, we will emphasise the fact that there is a large variety of STDP rules and also cite these articles.\n\n- **\"It is also unclear what results could be derived using the previous methods mentioned in lines 35–37\":**\n\n    This is closely related to the comment of Referee jhz1 on \"Mapping between spike and rate models\". We refer to this comment for a detailed response. \n    \n\n-  **Novelty and significance** (\"Convergence of a specific family of STDP rules to an already known regime ... constitutes ... a limited theoretical contribution\"):\n\n    We feel that the original article did not put sufficient emphasis on the main findings and the key contributions of this submission.\n\n  \n    * **Surprising convergence behaviour:**\n    Despite the constant injection of noise, the considered Hebbian learning dynamic converges to a point. This is non-standard and surprising as typically noisy gradient descent with fixed noise level only converges to a stationary regime where the noise causes the dynamic to fluctuate around a minimiser. This point will be stressed more in the revision.\n\n        Based on the analysis, we can show an exponentially fast convergence to the limit and quantify the influence of individual parameters such as the number of input neurons and the learning rate. The exponentially fast convergence is surprising as noisy stochastic gradient descent typically only converges with at most a polynomial rate, see e.g.  Mertikopoulos et al. (reference [23]) [for gradient descent exponentially fast convergence holds in some regimes].\n \n    * **Mathematical analysis of non-convex noisy SGD:** For convex loss functions, (noisy) SGD  is well-understood. The hardness of the considered learning rule is the underlying non-convex landscape with several stationary points. Analysing noisy SGD in non-convex settings is a notoriously hard problem. Our analysis is inspired by the recent work by Mertikopoulos et al.\\ (reference [23]) that previously appeared in NeurIPS. (In the past years several papers on theory for SGD in non-convex settings appeared in NeurIPS, including Patel et al. \"Global convergence and stability of stochastic gradient descent\" and Hu et al. \"Efficiency ordering of stochastic gradient descent.\")\n\n     - **Connecting recent theory of SGD with Hebbian learning rules:** To the best of our knowledge, this is the first article that provides a rigorous mathematical convergence analysis of a stochastic Hebbian learning rule. (For a comparison with Kempter et al. '99, see the response to Reviewer 2zhK). Cross-fertilisation between the theoretical learning community and practitioners has been proven to be extremely useful for the development of deep learning (highlights include the neural tangent kernel (NTK) that allows to understand gradient descent (GD) for wide networks, the fact that GD converges to weight balanced solutions, and the benign overfitting phenomenon that rigorously proves why GD learned fits can still generalise well in highly overparametrised models]). We envision a similar development  can likely generate new insights into Hebbian learning.  \n\n    Same as for SGD analysis for artificial neural networks, a rigorous theory requires to work in idealised models. In deep learning theory, the theoretical findings still align with empirical findings. \n\n    To extend the setting, we will also discuss several extensions to time-inhomogeneous intensities and correlated input neurons in the revised version. For details, please see the response \"2.1 Generalisation to non-homogeneous Poisson process\" (Reviewer 2zhK).\n\n- **Connection to noisy mirror descent:** We do believe that the Riemannian gradient flow, as derived in Section 3, is the more natural setting, although it leads to similar results. The loss function becomes, for instance, much simpler, see also the response to the next comment. \n    \n    Stochastic mirror descent is well-explored in the optimisation literature, both from its theoretical foundations and its implementation. The connection laid out in Chapter 4 enables the transfer of these insights to STDP. Potential implications for STDP and training SNNs include optimal choice of the hyperparameters (e.g., learning rate), but it is also conceivable that new variants of STDP can be introduced based on stochastic mirror descent insights.  \n\n    That the choice of the metric matters, has also been observed in the neuroscience literature, see e.g. Surace et al. (2020). (This point has been  also highlighted by Reviewer 2zhK).\n\n- **Intuitive interpretation for the loss function in Eq. 5:** We do not have a full answer to this question. However, here the connection to the natural gradient descent in Chapter 3 helps. Indeed in the natural geometry, the loss is the negative of a least squares loss and thus an energy term.\n\n\n- **Extension to several readout neurons:**\n\n    We can provide convergence guarantees for a slightly modified learning algorithm for multiple output neurons, where the output neurons are trained in consecutive episodes, each with $K$ training iterations. The first output neuron is trained as in Eq. (1), such that the convergence guarantee of Theorem 2.1 holds. After $K$ training steps, the weight $\\mathbf{w}_1$ is projected in the direction of the closest unit vector, measured in cosine similarity, and the learning is stopped. This yields the weight vector $\\mathbf{w}_1^\\ast =\\Vert\\mathbf{w}_1^\\ast\\Vert\\mathbf{e}_1$ with high probability. In the second episode, the second output neuron is trained. In the first step, we remove the component of the initial weight vector $\\mathbf{w}_2(0)$ into the direction of $\\mathbf{w}_1^\\ast$  ($\\mathbf{w}_2(0)\\gets \\mathbf{w}_2(0)-\\mathbf{w}_1^\\ast\\mathbf{w}_2(0)^T\\mathbf{w}_1^\\ast/\\Vert\\mathbf{w}_1^\\ast\\Vert$), effectively setting its first component to zero. The training proceeds as in Eq. (1), with now the second component dominating.  Again Theorem 2.1 applies. Proceeding iteratively for $d$ episodes, we obtain the weight matrix $\\mathbf{W}^\\ast$ and the corresponding probability matrix $\\mathbf{P}^\\ast$ with the following mathematical guarantee:\n\n    **New result:** For positive minimal gap at initialisation $\\Delta =\\min_{i=1,\\dots, d-1}([\\mathbf{p}\\_i(0)]\\_i - \\max_{j> i}[\\mathbf{p}_i(0)]\\_j)>0,$ \n\n  \\begin{equation*}\n    \\mathbb{P}(\\mathbf{P}^\\ast \\ \\text{is identity})\\to 1\n  \\end{equation*}\n  as $K\\to\\infty$.\n  More precisely, for $\\kappa := \\min_{i=1,\\dots, d}\\lambda_i/\\max_{i=1,\\dots, d}\\lambda_i$ and $\\epsilon>0$,\n\n  \\begin{equation*}\n        \\mathbb{P}(\\mathbf{P}^\\ast \\ \\text{is identity})\\ge (1-\\epsilon)^d \\quad\\text{for all }K> \\frac{16d}{\\alpha\\Delta(4+d\\Delta)}\\log\\Big(\\frac{4(1+\\kappa)}{\\epsilon\\kappa} \\Big).\n  \\end{equation*}\n    \n     \n- **Practical significance of Theorem 2.2 for training SNNs with STDP rules:**\n\n    Theorem 2.2 is the main result of this manuscript, establishing exponentially fast convergence ($=$linear  convergence in the optimisation literature) for the learning rule (3). \n\n    As mentioned above, there are two surprising findings, namely the fact that despite the constant injection of noise, the dynamic converges to a deterministic limit and secondly, the exponentially fast decay that is entirely unexpected for noisy SGD schemes. \n\n    The practical significance is that one does not have to worry about the noise and that one can expect an extremely fast convergence. The result also allows to pick a constant learning rate, quantifies the range of possible learning rates, and helps to understand the influence of other parameters in the model.\n\n- **Which known STDP rules fall under the formulation of Eq. (1), and which do not?**\n\n    The considered learning rules are based on the pair-based STDP models for excitatory neurons as discussed in [11], Section 19.2.2 with amplitude parameters $A_+(w)=A_-(w)=w.$ As detailed in the first response to Reviewer jhz1, we closely mimic the distribution of the original postsynaptic firing times.\n\n    We will make this very clear in the revised version. \n\n- **Minor comments:** They will be addressed in the revised article. Thank you for your suggestions."}, {"review_id": "DtWYxgjCwc", "reviewer": "Reviewer_jhz1", "summary": "The paper studies spike-timing dependent plasticity (STDP) in a feedforward network of d input neurons projecting on a single postsynaptic neuron. Rather than analyzing  the classical STDP model (Gerstner 1996), which the authors argue is complex to analyze, the authors consider a simplified version of it, which is described in section 2.3. The authors make a thorough analysis of this simplified model, showing convergence by casting its dynamics as noisy gradient descent of a specific loss function. Notably, this simplified model behaves as a winner-take-all selection mechanism, with the postsynaptic neuron aligning with the highest firing input neuron. This is a departure from the rate case (e.g. Oja’s rule) in which the post-synaptic neuron aligns with the first principal component of the input layer.", "questions": "1/ Could the authors explain how their simplified model incorporates the integration behavior of standard STDP? E.g. how are the postsynaptic spike times dependent on the threshold S?\n\n2/ A classic result for rate models, that the authors point to, is that rate models under with Hebbian plasticity learn to extract the principal components of the input data. Often there is a somewhat natural mapping between spike and rate models, specifically STDP and Hebbian learning, that would bridge the existing PC results to the authors’ results regarding the alignment of the output neuron to the highest firing input neuron direction. This is interesting. Do the authors have any suggestions about why that does not appear to be the case here?", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n- The math and presentation of the derivations are very rigorous and well-explained.\n- Convergence of this model is rigorously established by casting it as noisy gradient descent.\n\nBroadly, the paper was a valiant effort at using sophisticated mathematical tools to analyse a system of neuroscientific interest. It performs a thorough theoretical analysis of a simplified model that is appreciated.\n\nWeaknesses:\n\nHowever, the value of the contribution is hindered slightly by the clarity of the presentation for neuroscientist, but majorly by the large leap required between STDP and the stated model.\n\n- Relevance of the simplified model to neuroscience: The jump from the classical STDP type model to the simplified model is worrying. A crucial part of the integrate-and-fire model is the fact that it integrates inputs in order to choose when to spike. On first reading, this appears to be completely missing from the simplified model. Further, the classic components of STDP, such as spike-time dependent kernels are also missing. These details all seem to be wrapped up in the procedure for selecting the presynaptic spike at which the postsynaptic neuron fires. However, this seemingly crucial selection procedure is somewhat swept under the carpet (line 236). The best version of this claim would be that you can choose this selection procedure in order to model a classical integrate-and-fire neuron with STDP. Yet no evidence is presented that this is possible. Providing such evidence would significantly strengthen the article. More broadly, more discussion regarding these differences between the toy and classical model is strongly recommended.\n\n- High level presentation: The paper seems formatted backwards in some ways. For example, the easier to understand analysis, on gradient flow, arrives after the complex discrete time analysis. Further, the movement from the classical integrate-and-fire model to the more toy model was presented after a long discussion of the toy model. For a neuroscientist the natural place to start would be the classical model, then explain the simplifications, before ending with the resulting analysis of the model.\n\nMore minor points:\n\n- Some figures are not informative: e.g. Fig 1: shows a feedforward network of spiking neurons, which is very common in the field, but gives no insights on the plasticity rule modeling. A cartoon would greatly help to ease the reader into the authors’ way of considering STDP.\n\n- The links to mirror descent could be interesting, but it is unclear how it is used to create actionable insights. It feels like it is another mathematical framework presented regarding dynamics that have the same equilibria, and the neuroscientifically interesting conclusion is the structure of these equilibria. What then do we gain via an alternative characterization of the dynamics? It would be good to signpost the role it is hoped that this additional analysis plays.\n\n- Typo in the appendix: the loss on line 769 is missing a square.", "was_revised": true, "final_justification": "The authors have clarified several points that made this work initially hard for me to appreciate, such as the connection to existing work and novelty. As a result, I have increased my score to 4.", "rebuttal": "Thank you very much for your assessment of our manuscript. We do think that despite the idealized model, this submission is of sufficient significance and we believe that we could address all your concerns. Feel free to provide us with additional feedback in case something is unclear or needs additional clarification. Thank you again!\n\n- **How does the model incorporate the integration behavior of standard STDP? E.g. how are the postsynaptic spike times dependent on the threshold S?**\n\n    We do argue that the postsynaptic spike times that we consider follow a similar, and in some cases even the same distribution, as the spike times determined by the threshold model. In particular, they depend on the threshold $S.$ Below, we sketch this argument. To clarify this point, we will also add it to the revised article.\n    \n    The common model is to assume that the postsynaptic neuron fires at the moment when the membrane potential exceeds the threshold $S.$ According to this rule, the postsynaptic spikes can only happen at moments when one of the presynaptic neurons spike. For convenience, let us assume that the previous postsynaptic spike happened at time $0.$ Denote by $\\tau_{j1},\\tau_{j2},\\ldots$ the spike times of the $j$-th presynaptic neuron after time $0$ in increasing order. \n\n    As the next postsynaptic spike $\\mathfrak{t}$ can only happen at a presynaptic spike time, the distribution of the postsynaptic spikes is completely determined by the probabilities\n    \n    $$P\\big(\\mathfrak{t}=\\tau_{j\\ell}\\big), \\quad j=1,\\ldots,d, \\quad \\ell=1,2,\\ldots$$\n    These probabilities do depend on the threshold $S.$\n\n    In the derivation of our learning rule, we select the postsynaptic spike times to match this distribution as closely as possible. For that, we rewrite the probability into \n    \n    $$P\\big(\\mathfrak{t}=\\tau_{j\\ell}\\big)\n        = P\\big(\\mathfrak{t}=\\tau_{j\\ell} \\big|\\mathfrak{t}\\in (\\tau_{jk})\\_{k\\geq 1} \\big)\n        P\\big(\\mathfrak{t}\\in (\\tau_{jk})\\_{k\\geq 1}\\big) $$\n\n    This means, in a first step, we need to select the probability $P(\\mathfrak{t}\\in (\\tau_{jk})\\_{k\\geq 1})$ that the $j$-th presynaptic spike train causes the next postsynaptic spike. In Section A.3, we show that the probabilities $\\widetilde P(\\mathfrak{t}\\in (\\tau_{jk})_{k\\geq 1})$ that we select nearly match the probabilities of the original dynamic. In particular, Lemma A.5 describes a setting, where this is even exact. \n\n    Our argument does not require a specific choice of the second probability $P(\\mathfrak{t}=\\tau_{j\\ell} |\\mathfrak{t}\\in (\\tau_{jk})_{k\\geq 1}).$ This means, here we can take the one from the original dynamic. This probability will also incorporate most of the dependence on $S.$\n\n    That the considered model is of integrate-and-fire type has also been acknowledged by Referee 2zhK. \n    \n- **Spike-time dependent kernels are missing:** Indeed, we work with a kernel that is a jump followed by an exponential decay (see also the report of Reviewer 2zhK). This is a common choice. The derivation of the learning rule (3) does not exploit the specific structure of the kernel. If you have a specific class of kernels in mind, we would be happy to include it in the revision. \n\n- **\"The paper seems formatted backwards in some ways... For a neuroscientist the natural place to start would be the classical model, then explain the simplifications, before ending with the resulting analysis of the model\":**  While the paper has been mainly evaluated from the neuroscience perspective, our aim is to contribute an article that is appealing to the whole NeurIPs community and in particular also researchers working on theoretical guarantees for SGD. Even in its current form, Referee vBKN suggests \"to target a slightly wider audience\". We do acknowledge that for neuroscience, a different structuring of the material might be more natural. To make the paper more accessible to the SGD community motivated the way we have presented the results, stating the idealized learning rule right at the beginning and deferring the rather technical derivation from standard STDP rules to a later chapter.\n\n    If our reasoning on this point is unclear or unconvincing, we are very open to suggestions. Should you and the other referees feel that a different structure would be more appropriate, we would of course be happy to make the necessary changes.\n    \n    \n- **Mapping between spike and rate models:**\n\n    It is known that rate-based learning rules will do some form of principal component analysis. While we take the stochasticity into account, rate-based models only capture the mean behaviour. This makes, however, a huge difference and can lead to different behaviour. An example is a zero-mean stochastic process: The rate-based behaviour is just the zero function (the mean of the process), whereas a sample path can be very wild and also diverge to infinity. \n\n    For instance, the rightmost plot in Figure 2 shows that due to randomness, the dynamic can end up in a different local minimum.\n\n\n    Rate-based models are obtained already by taking a limit $t\\to \\infty.$ This means that they operate effectively on a slower time scale. Thus, the exponential fast decay of the STDP rule suggests a much faster convergence than an exponential decay in a rate-based model.\n\n    An exception to the analysis of rate-based models is Kempter et al. '99 which also includes noise but rewrites the dynamic into a deterministic ODE. The analysis and the results are both different. For more details on that, please see the first point in the reply to 2zhK.\n\n    We will add the argument above to the revised version.\n\n- **More minor points:** In the revised version, we will add some information on the learning dynamic to Figure 1. Regarding the connection to mirror descent, please see the reply to Referee 9Yjy. Thank you for spotting the typo in line 769."}, {"review_id": "qfcCBgULqM", "reviewer": "Reviewer_vBKN", "summary": "In this work, the authors establish a mathematical connection between spike-timing-dependent plasticity (STDP) and noisy gradient descent on a probability simplex. The nature of this connection allows for mathematical analysis which goes beyond previous techniques which utilized ensemble averages. The main finding of their analysis is that their STDP learning rule converges exponentially fast to identify the presynaptic neuron with the highest firing rate. Additionally, they establish a connection to noisy mirror descent, specifically entropic gradient descent.", "questions": "Regarding the selection probabilities (section beginning line 238), I would assume that, under most biological conditions, a handful of presynaptic spikes (on the order of tens) would be sufficient to trigger a postsynaptic spike. Is there any perturbation or corrective term which can be added to move even slightly away from the two limit cases (“weights are very small” or “weights are large enough to for any presynaptic activity to trigger a postsynaptic spike”)? \n\nI lost the plot a bit around theorem 2.2. I was able to answer the questions I had by going to the appendix, but it broke up the flow of the paper a bit for me. Perhaps the authors can consider adding more “connective tissue” in the main text to guide the reader, some bits are currently limited to an expert audience.  \n\nI would be curious to see a more direct comparison to the previous methods of STDP analysis which used ensemble averages.", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "This work provides a novel analysis of STDP, one that is certainly significant to the fields of computational neuroscience and AI. The mathematical formulation linking local spike-based updates to global optimization is valuable. The  mirror descent perspective is of particular interest, connecting STDP to an intriguing framework from Cornford et al. 2024. The assumptions and proofs are well described, though it may benefit the authors to target a slightly wider audience. While the analysis is novel and interesting, it does require some assumptions which limit its applicability to STDP in biological conditions (the authors acknowledge these limitations).", "was_revised": true, "final_justification": "This paper provides a new take on STDP with strong theoretical backing.", "rebuttal": "Thank you for your positive assessment of our work. We fully agree that for NeurIPS the manuscript should be readable by a wide audience. \n\n\n- **\"it may benefit the authors to target a slightly wider audience\":**\n\n    As mentioned also in the response to Reviewer jhz1,  our aim is to contribute an article that is appealing to the whole NeurIPS community and in particular also researchers working on theoretical guarantees for SGD. \n    \n    Based on all the received feedback, we will be able to add some discussion that is beneficial to a general readership. If you have specific suggestions, we would of course be very happy to learn about them. \n    \n- **\"Is there any perturbation or corrective term which can be added to move even slightly away from the two limit cases\":**\n    \n    In general it is hard to find an exact representation of the probabilities beyond the cases already stated in the paper, see Novikov et al. 2005 for a broader discussion.\n    \n    However, the formula for the probabilities can also be motivated by the following heuristic. The postsynaptic membrane potential $Y$ grows, in expectation, at an approximately linear rate $\\mathbf{\\boldsymbol{\\lambda}^\\top \\mathbf{w}}$. Furthermore, input $i$ causes a postsynaptic spike if, and only if, it spikes at a time at which $Y\\geq S-w_i,$ where $S>0$ is the threshold level. As $Y$'s growth is approximately linear with rate $\\mathbf{\\boldsymbol{\\lambda}^\\top \\mathbf{w}}$, the amount of time in which $Y\\geq S-w_i,$ holds is approximately $w_i/\\mathbf{\\boldsymbol{\\lambda}^\\top \\mathbf{w}}$. Now the probability that input $i$ jumps in an interval of length $w_i/\\mathbf{\\boldsymbol{\\lambda}^\\top \\mathbf{w}}$ is given by $1-\\exp(-\\lambda_iw_i/\\mathbf{\\boldsymbol{\\lambda}^\\top \\mathbf{w}})\\approx \\lambda_iw_i/\\mathbf{\\boldsymbol{\\lambda}^\\top \\mathbf{w}},$ which is our modelling choice in the independent setting.\n\n    We will add this heuristic to the revised manuscript. \n    \n- **Adding more material around Theorem 2.2:** Of course, we could add more discussion here in the revised version. For us it would be very valuable to learn what specifically you missed in the text. \n\n* **A more direct comparison to the previous methods of STDP analysis which used ensemble averages:** This is closely related to the second comment of the first referee and one comment of the second referee. We reproduce the answer below: \n\n    It is known that rate-based learning rules will do some form of principal component analysis. While we take the stochasticity into account, rate-based models only capture the mean behaviour. This makes, however, a huge difference and can lead to different behaviour. An example is a zero-mean stochastic process: The rate-based behaviour is just the zero function (the mean of the process), whereas the sample path can be very wild and also diverge to infinity. \n\n    For instance, the rightmost plot in Figure 2 shows that due to randomness, the dynamic can end up in a different local minimum. Rate-based models are obtained already by taking a limit $t\\to \\infty.$ This means that they operate effectively on a slower time scale. Thus, the exponential fast decay of the STDP rule suggests a much faster convergence than an exponential decay in a rate-based model.\n\n    An exception to the analysis of rate-based models is Kempter et al. '99 which also includes noise but rewrites the dynamic into a deterministic ODE. The analysis and the results are both different. For more details on that, please see the first point in the reply to 2zhK.\n\n    We will add the argument above to the revised version.\n\n\n\n\nNovikov et al. (2005) Prob Eng Mech doi:10.1016/j.probengmech.2004.04.005"}, {"review_id": "EL8APC9XVn", "reviewer": "Reviewer_2zhK", "summary": "In this manuscript, the authors return to classic pair-based STDP models, in which the change of a synapse depends on the relative timing of the spikes of the pre- and postsynaptic neurons. The authors prove mathematically that this type of STDP model identifies the pre-neuron with the highest triggering probability. Moreover, they show that it is connected to noisy mirror descent.", "questions": "Please see \"Strengths And Weaknesses\" for my concern and questions.", "limitations": "No. The authors have not adequately addressed the limitations of their work. While the theoretical analysis is presented clearly, the manuscript overlooks several critical limitations that should be acknowledged and discussed:\n\n- Connection to Prior Work: The authors are not aware of important, closely related literature, which limits the novelty of their results. (Please see \"Strengths And Weaknesses\" for more detail.)\n\n- Narrow Scope of Input Models: The work assumes independent, homogeneous Poisson inputs—a highly restrictive and biologically unrealistic setting. This limits the applicability of the results. The potential for generalization to inhomogeneous or correlated inputs should be addressed.\n\n- Simplifying Assumptions in STDP Rule: The analysis is restricted to a specific, highly idealized form of pair-based STDP—likely the balanced, antisymmetric case. The authors should clarify whether their results hold for more general STDP methods.\n\n- Biological Plausibility: The model assumes pair-based STDP, which has been shown to be an incomplete description of biological plasticity. Experimental studies (e.g., Sjöström et al., 2001; Clopath et al.) have demonstrated the importance of triplet interactions and voltage dependence. These factors should at least be acknowledged as limitations of the modeling framework.", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 4, "significance": 2, "originality": 2, "strengths_and_weaknesses": "In the literature (reviewed in the cited paper of Morrison et al.), there are many versions of such a pair-based STDP rule. If I understand correctly, the authors focus on the one where, for the sequence post-before-pre and pre-before-post, only the closest postsynaptic spike matters, whereas all presynaptic spikes between two postsynaptic spikes are taken into account. This assumption is well accepted, and I don't criticize it.\n\n\nExisting analytical models of STDP (e.g. Kempter et al. 1999 as cited) have mostly assumed stochastic spike arrival with Poisson rate $\\lambda_j$  and stochastic output spiking with instantaneous rate (conditional stochastic intensity)  $\\lambda_{post} = \\sum_j w_j \\int_0^\\infty \\epsilon(s) \\lambda_j(t-s) ds$ with some integration kernel $\\epsilon$. They found that pair-based STDP performs PCA. For uncorrelated Poisson input, there is a positive feedback loop proportional to the overlap\n$\\int_0^\\infty  W(s) \\epsilon(s) w_j \\lambda_j ds$ where $W(s)$ is the causal (=pre-before-post) section of the STDP window.\n\nThe authors modify this standard paradigm a little bit in that they assume that the postsynaptic neuron has no intrinsic noise. Instead, output spikes are defined by a deterministic threshold crossing process (leaky integrate-and-fire, LIF). Under the assumption (i) that the kernel $\\epsilon$ is a jump followed by exponential decay, and (ii) that the stochastic firing intensity of presynaptic neurons is constant, the authors correctly observe that the threshold crossing occurs at the moment of (or arbitrarily shortly after) presynaptic spike arrival and that the relative probability that a presynaptic neuron j is the one that triggers the postsynaptic spike is (if all weights are small)\n\n$$p(j) = \\frac{\\lambda_j w_j}{\\sum_l \\lambda_l w_l}.  (Eq. 12)$$\n\nI like this observation, and it looks novel to me!\nFrom this observation, together with the assumption that a sequence pre-before-post leads to weight increase, it follows immediately that the input weight vector becomes in the limit parallel to $p(j)$. Indeed, the resulting self-feedback term is very close to that of Kempter et al. (1999). The difference is that in the noisy neuron model of Kempter, the full time course of $\\epsilon$ plays a role, whereas in the deterministic LIF model, only the initial moment of the rapid rise of $\\epsilon$ plays a role. Indeed, it is known in computational neuroscience that the threshold process of an LIF generates correlations that are sensitive to the derivative of $\\epsilon$.\n\nStarting from Eq. 12 (which is unfortunately only given in Section 2.3), the authors have arrived at a formal mathematical model that distinguishes the input spike that triggers the postsynaptic spike from all other input spikes. This model enables mathematical analysis. As expected from earlier work, weights explode, and the weight vector becomes aligned with the strongest input.\n\nIn the formal mathematical model (from line 225), the algorithm \"selects a presynaptic spike time $\\tau^* > t_k$ and then sets $t_{k+1} = \\tau^∗$. This is done in two stages by first choosing a presynaptic neuron $j^∗(k + 1) \\in [d]$ and, in a second step, picking a spike time $τ^∗$ among the spike times of the $j^∗(k + 1)^{th}$ presynaptic neuron.\" I would have preferred to see first the biological motivation of Section 2.3. before the abstract mathematical model, but this is a matter of taste and only a minor point.\n\n----\n\n**Major**:\n\n1. The authors are not aware of important closely related literature  which limits the novelty of their results. Between 1996 and 2007, (pair-based) STDP attracted a lot of research interest.\n\n1.1 Kempter et al. (1999, cited) already analyzed the evolution of synaptic weights in a mathematical model where the (stochastic) generation of output spikes depend linearly on the input spikes. Even though the noise model is different from that of the current manuscript, the results are essentially the same. Importantly, Kempter et al. (1999) cover a much broader range of input paradigms that go well beyond the independent homogeneous Poisson processes considered in the current manuscript.\n\n1.2. As part of the efforts 20 years ago, several loss functions have been proposed that yield STDP via gradient descent. Noteworthy examples are Chechik 2003, Bohte and Moser 2005, Bell and Para 2005, Toyoizumi et al. 2005; Toyoizumi et al. 2005a, Pfister et al. 2006, Toyoizumi et al. 2007.\n\n1.3. The general theoretical framework of gradient descent in computatonal neuroscience has been reviewed by Surace et al. 2020 who highlight the importance of a metric in order to fully define gradient descent. The metric of mirror descent is one example. A rescaling of parameters from a to b=exp(a) to guarantee positive values of b is a standard trick in the field.\n\n2. It is unfortunate that the authors consider only a very specialized stimulation paradigm and a very specific case of STDP.\n\n2.1. I believe it would be possible to generalize the approach (i)  from homogeneous to inhomogeneous Poisson input using the time rescaling theorem and (ii) from independent to correlated Poisson inputs. For the latter, several possibilities exist to generate correlations in the input. Note that Kempter et al. (1999) already analyzed for their neuron model a rather general case of inputs. They indeed found that STDP generated PCA as conjectured in the manuscript.\n\n2.2 In standard pair based STDP there are four parameters, i.e., two time constants and two amplitude parameters. There is a well-known special case of STDP which is called balanced STDP. In this special case the total area under the STDP window is zero. If, in addition you require that the time constants for pre-before-post and post-before-pre are identical, then we get an exactly antisymmetric STDP window.  If I am not mistaken, the authors make both assumptions (integral zero and same time constant) to get their Eq. (11). How would the results generalize to other cases?\n\n2.3. Unfortunately, pair-based STDP is NOT a good representation of biological data as shown by Sjostrom et al. (2001). First, more than two spikes interact ('triplet effect') and second, the voltage at the location of the synapse also plays a role (for both, see e.g.  the cited paper of Clopath et al). These observations  might have contributed to the reduced interest in computational neuroscience for standard pair-based STDP models even though they are still widely used.\n\n\nAdditional references:\n\n\nSjostrom, P., G. Turrigiano, and S. Nelson (2001). Rate, timing, and cooperativity jointly determine cortical synaptic plasticity. Neuron 32, 1149–1164.\n\nChechik, G. (2003). Spike-timing-dependent plasticity and relevant mututal information\nmaximization. Neural Computation 15, 1481–1510\n\nBohte, S. M. and M. C. Mozer (2005). Reducing spike train variability: A computational\ntheory of spike-timing dependent plasticity. In L. K. Saul, Y. Weiss, and L. Bottou\n(Eds.), Advances in Neural Information Processing Systems 17, pp. 201–208. Cam-\nbridge, MA: MIT Press\n\nBell, A. J. and L. C. Parra (2005). Maximising sensitivity in a spiking network. In L. K.\nSaul, Y. Weiss, and L. Bottou (Eds.), Advances in Neural Information Processing\nSystems 17, pp. 121–128. Cambridge, MA: MIT Press.\n\n\nT. Toyoizumi, J.-P. Pfister, K. Aihara, and W. Gerstner (2007)\nOptimality Model of Unsupervised Spike-Timing Dependent Plasticity: Synaptic Memory and Weight Distribution\nNeural Computation, 19: 639-671\n\n\nT. Toyoizumi, J.-P. Pfister, K. Aihara, and W. Gerstner (2005)\nGeneralized Bienenstock-Cooper-Munro rule for spiking neurons that maximizes information transmission\nProc. Natl. Acad. Sci. USA, 102:5239-5244\n\nToyoizumi, T., J.-P. Pfister, K. Aihara, and W. Gerstner (2005a). Spike-timing dependent\nplasticity and mutual information maximization for a spiking neuron model. In L. K. Saul, Y. Weiss, and L. Bottou (Eds.), Advances in Neural Information Processing Systems 17, pp. 1409–1416. Cambridge, MA: MIT Press.\n\nJ.-P. Pfister, T. Toyoizumi, D. Barber, and W. Gerstner (2006)\nOptimal Spike-Timing Dependent Plasticity for Precise Action Potential Firing in supervised learning\nNeural Computation 18:1309-1339\n\nT. Toyoizumi, J.-P. Pfister, K. Aihara, and W. Gerstner (2007)\nOptimality Model of Unsupervised Spike-Timing Dependent Plasticity: Synaptic Memory and Weight Distribution\nNeural Computation, 19: 639-67\n\nS.C. Surace, J.-P. Pfister, W. Gerstner, and J. Brea (2020)\nOn the choice of metric in gradient-based theories of brain function\nPLoS ComputBiol 16(4):e1007640,\ndoi: 10.1371/journal.pcbi.1007640", "was_revised": true, "final_justification": "As noted in my review, I recommend borderline acceptance to NeurIPS.", "rebuttal": "This is a truly impressive report. The level of your expert knowledge of the field and the careful analysis of our submission is outstanding and has helped us a lot to position our work. We are especially grateful for the suggestions regarding related articles and will revise the paper to include a broader comparison with the existing literature. Overall, we do believe that we could address your comments.\nWe would be delighted to receive more feedback during the forthcoming discussion process.\n\n- **1.1 Comparison with Kempter et al. 1999:**\n\n    Kempter et al. 1999 start with a noisy spike-time dependent dynamic which is transformed into a deterministic ODE by imposing a slow learning rate and using the self-averaging effect of the system. The important correlations between input and output spike sequences are encapsulated via a covariance type matrix. The ODE is analysed. The advantage of this approach is that a wide range of choices regarding the structure of the input can be made. In particular, one can include time-inhomogeneous intensities and correlations. \n    \n    One major difference is the influence of the noise. In their work, the variance grows linearly in $t$ and a careful comparison of time scales is required. In our manuscript, despite a constant injection of noise into the system, the dynamic converges to a deterministic limit. (The comparison is more subtle as they study the weights and we study the dynamic of the triggering probabilities.) The second major difference is that we track the influence of the realised noise in every step and use recent ideas from the analysis of noisy SGD to prove convergence. We do acknowledge that a more detailed comparison was missing. This will be added to the revised version. \n\n- **1.2 Previously considered loss functions that yield STDP via gradient descent:** \n\n    As mentioned, this line of work starts with a loss function and then derives a form of gradient descent, while our approach starts with pair based STDP and derives from there a stochastic learning rule and an associated loss function. We will cite these articles in the revision and acknowledge that the connection between learning and gradient descent has been integral to prior work. In the revision, we will also compare to Scellier and Bengio '17. \n\n- **1.3 Comparison with Surace et al. 2020:** Thank you for pointing us to this article that we overlooked. While we already cite Cornford et al., in the revision we will emphasize that there has been more previous work in neuroscience highlighting that the Euclidean gradient might not be the most natural choice and mentioning the Fisher information metric in this context. We will moreover cite Neumann et al.\\ \"Intrinsic plasticity via natural gradient descent with application to drift compensation\" (2013). The sentence \"We also discover an intrinsic connection to noisy mirror descent\" will be removed from the abstract. \n\n- **2.1 Generalization to non-homogeneous Poisson process and correlated inputs:** Thank you for asking about time-dependent intensities. In the revised version, we will add a small section deriving the following results:\n\n\n    **(a)**  Small perturbations of the intensity will not affect the convergence result. In fact in every step the intensities can vary by (small constant)$\\times \\alpha \\Delta$ ($\\alpha$ the learning rate, $\\Delta$ the separation quantity) without changing the limit. \n\n\n     **(b)** Large changes in the intensities can of course lead to different limits and it is important in which order the intensities are presented to the learning rule. The next result (which we include in the revision) shows that the dynamic can be primed at the beginning to end up in one regime. Despite the noise and the infinite amount of data, the dynamic is unable to escape the domain of attraction.\n\n    **New result:** Given an initialization of the weights $(w_1(0),\\ldots,w_d(0))^\\top$ consider two $d$-dimensional intensity vectors $(\\lambda_1,\\ldots,\\lambda_d)^\\top, (\\widetilde \\lambda_1,\\ldots, \\widetilde \\lambda_d)^\\top$ with positive entries. Assume that $\\lambda_1 w_1(0)> \\max_{i=2,\\ldots,d} \\lambda_i w_i(0)$ and $\\widetilde \\lambda_d w_d(0)> \\max_{i=1,\\ldots,d-1} \\widetilde \\lambda_i w_i(0).$ Assume we run the learning rule (3) with intensity $\\boldsymbol{\\lambda}$ until time $K^* $ and then, for $k>K^* $, change the intensity to $\\widetilde{\\boldsymbol{\\lambda}} $ and run the learning dynamic until time $k\\to \\infty.$ For any $\\epsilon\\in (0,1)$ and all sufficiently large $K^*$, the dynamics will converge to $\\mathbf{e}_1$ with probability $1-2\\epsilon.$\n\n    \n    From the proof, one can derive quantitative bounds for $K^*.$\n\n\n    **(c)** We will also derive the gradient flow version in the time-inhomogeneous case\n        \\begin{align*}\n        \\dot{\\mathbf{p}} =\\mathbf{p}\\odot\\left(\\dot{\\log(\\boldsymbol{\\lambda})}-\\mathbf{p}^T \\dot{\\log(\\boldsymbol{\\lambda})}\\mathbf{1}\\right)+\\mathbf{p}\\odot(\\mathbf{p}-\\Vert\\mathbf{p}\\Vert^2\\mathbf{1}).\n    \\end{align*}\n    Here, $\\boldsymbol{\\lambda}$ is a vector-valued function, $\\log()$ has to be interpreted componentwise, and the dot denotes the time derivative.\n\n    **(d)**  We agree that the setting of independent inputs is too restrictive. In order to introduce correlations between the inputs, we propose to allow the inputs to jump simultaneously. The probability that input $i$ and $j$ jump at the same time is denoted by $\\rho_{ij},$ and naturally $\\rho_{ii}=1,$ for all $i\\in[d]$. For $i,j\\in[d],k\\in\\mathbb{N},$ we introduce independent random variables $c_{i,j}(k),$ such that $c_{i,j}(k) \\sim \\operatorname{Ber}(\\rho_{i,j}),\\  i\\leq j\\in[d],$ and we set $c_{j,i}:=c_{i,j}\\  i\\leq j\\in[d],$ and $\\mathbf{C}(k):=(c_{i,j}(k))_{ij}.$ \n\n    The random variables $Z$ then stay centered, however the behaviour of the $\\mathbf{B}$'s changes, because multiple inputs can jump at a postsynaptic spike. The probability of $B_i(k)$ being one is now given by $\\boldsymbol{\\rho}\\_{i,\\cdot} \\mathbf{p}(k), $ where $\\boldsymbol{\\rho}=(\\rho_{ij})\\_{ij} $ is a $d\\times d$ matrix and $\\boldsymbol{\\rho}_{i,\\cdot}$ denotes the $i$-th row. Since multiple input neurons can spike simultaneously we adjust the modelling of the probabilities $\\mathbf{p}$ to\n \n  \\begin{align*}\n    p_i(k)=\\frac{\\lambda_i(w_i(k) + \\sum_{j,j\\neq i}c_{i,j}(k)w_j(k))}{\\boldsymbol{\\lambda}^\\top \\mathbf{w}(k) + \\sum_{i,j,i\\neq j}\\lambda_ic_{i,j}(k)w_j(k)} = \\frac{\\lambda_i\\boldsymbol{C}_{i,}(k)\\mathbf{w}}{\\boldsymbol{\\lambda}^\\top \\boldsymbol{C}(k) \\mathbf{w}},\n  \\end{align*}\n   that is, the weight corresponding to each jump in the independent setting is replaced by the effective weight in which all inputs are taken into account. It holds\n\n  \\begin{equation*}\n    \\mathbf{p}(k+1) = \\frac{\\boldsymbol{\\lambda} \\odot (\\mathbf{C}(k+1) \\mathbf{w}(k+1))}{\\boldsymbol{\\lambda}^\\top \\mathbf{C}(k+1)\\mathbf{w}(k+1)} = \\frac{\\boldsymbol{\\lambda} \\odot (\\mathbf{C}(k+1) (\\mathbf{w}(k)\\odot(\\mathbf{1}+\\alpha\\mathbf{Y}(k))))}{\\boldsymbol{\\lambda}^\\top \\mathbf{C}(k+1)(\\mathbf{w}(k)\\odot(\\mathbf{1}+\\alpha\\mathbf{Y}(k))}.\n  \\end{equation*}\n  Since $\\mathbb{E}[\\mathbf{Y}(k)]=\\boldsymbol{\\rho}\\mathbf{p}(k), \\mathbb{E}[\\mathbf{C}(k)]=\\boldsymbol{\\rho}$, we obtain the gradient flow \n\n  \\begin{equation*}\n    \\frac{\\mathbf{d}}{\\mathbf{d} t}\\mathbf{p}(t) = \\mathbf{p}(t)\\odot (\\boldsymbol{\\rho}\\mathbf{p}(t) - \\mathbf{p}(t)^\\top \\boldsymbol{\\rho}\\mathbf{p}(t)\\mathbf{1}),\n  \\end{equation*}\n  which again is a replicator equation, and the Shahshahani-loss is $\\mathbf{x}\\mapsto \\frac{1}{2}\\mathbf{x}^\\top \\boldsymbol{\\rho}\\mathbf{x}$. A rigorous mathematical analysis of this correlated model follows the steps in the independent case.\n\n- **2.2 Extension to non-balanced STDP:**\n\n    Our approach can be adjusted to non-balanced STDP in the following way. Let $W(x)$, $x\\in\\mathbb{R}$, be the learning window let $\\Delta\\tau$ be a generic difference between a pre- and a postsynaptic spike. Define the ensemble average $\\bar{W}=\\mathbb{E}[W(\\tau)]$, for example $\\bar{W}=\\int W(x)d x$ for uniformly distributed jump times within the support of $W$. We adjust the original learning rule (Eq. 1) such that\n    \\begin{equation*}\n        \\mathbf{w}(k+1) = \\mathbf{w}(k)\\odot \\left(1 + \\alpha(\\mathbf{B}(k) + \\mathbf{Z}(k) - \\bar{W}\\mathbf{1})\\right).\n    \\end{equation*}\n    Since $\\mathbb{E}[\\mathbf{Z}(k)]=\\bar{W}\\mathbf{1}$, we have for $\\tilde{\\mathbf{Y}}=\\mathbf{B}+\\mathbf{Z}-\\bar{W}\\mathbf{1}$ that $\\tilde{\\mathbf{Y}}$ is bounded and that $\\mathbb{E}[\\tilde{\\mathbf{Y}}\\vert\\mathbf{p}(k)]=\\mathbf{p}(k)$. These two properties allow us to extend the convergence guarantees as in Theorem 2.1 to non-balanced STDP. The subtle interplay between the integral of the learning window and the linear term $\\bar{W}$ has already been observed by Kempter et al 2001.\n\n- **2.3 Beyond pair based STDP:** In the revised version, we will highlight the limitations of pair-based STDP rules in the discussion section and cite Sjostrom et al. (2001). We will also briefly discuss extension to triplet and voltage-dependent rules. \n\n**Questions:** The questions are linked to the previous comments.\n\n- **Connection to Prior Work:** While we tried to provide a summary of the previous work, we sometimes only mentioned articles such as the related Kempter et al. paper without a more detailed comparison. We do acknowledge that this needs to be improved in the revised version. \n    \n    The comment refers to the lines of work mentioned in 1.1-1.3 above. Please see the specific answers to those comments.  \n\n- **Narrow scope of input models:** Please see the response above on 2.1 generalization to non-homogeneous Poisson processes and correlated inputs.\n\n- **Simplifying assumptions in STDP rule/ Extension to non-balanced STDP:** Please see our response to 2.2. \n\n- **Biological plausibility/ Beyond pair-based STDP:** Please see our response to 2.3.\n\nKempter et. al (2001) Neural Comput\n\nScellier, Bengio (2017) Front Comput Neurosci"}]

Author rebuttal:

Meta-review: This paper gives a rigorous, optimization-theoretic account of a class of spike-timing–dependent Hebbian rules by recasting them as noisy gradient descent on an associated loss. Within the setting of feedforward network with a single output neuron, the authors prove convergence of the spike-triggering probability to a winner-take-all solution that selects the most active presynaptic neuron, with linear rate despite persistent noise; they also elucidate a connection to noisy mirror descent. Reviewers found the derivations careful and the result conceptually clarifying, contrasting it with rate-based analyses that typically yield PCA-like behavior. The work’s principal strengths are (i) a clean formulation that links local STDP updates to global optimization geometry; (ii) a rigorous convergence guarantee for a stochastic, non-convex learning dynamic; and (iii) a bridge between computational neuroscience and modern optimization language that could seed follow-ups in SNN training.

The main weaknesses raised were scope and positioning: reliance on an idealized pair-based STDP and simplified spike-selection mechanism; initially narrow input assumptions (independent, homogeneous Poisson); ambiguity about what existing STDP variants are covered; limited discussion of prior STDP-as-optimization literature; and presentation/ordering that made accessibility difficult for non-specialists. During rebuttal, the authors substantially strengthened the case: they clarified the mapping from classical integrate-and-fire STDP to their two-stage selection rule and its dependence on threshold, specified the family of pair-based rules encompassed, and extended the analysis to multiple readout neurons via episodic training, to time-inhomogeneous and correlated inputs, and to non-balanced windows; they also broadened related-work coverage and committed to clearer exposition. These responses led all reviewers to settle at borderline-accept or accept. I recommend acceptance.
Decision: Accept (poster)