Formalizing locality for normative synaptic plasticity models
Authors: Colin Bredenberg
Ezekiel Williams
Cristina Savin
Blake Aaron Richards
Guillaume Lajoie
Venue: NeurIPS 2023 poster (2023)
Keywords: synaptic plasticity
computational neuroscience
TL;DR: We provide a formal framework for discussions of locality in normative plasticity models, fit existing models into this framework, and discuss implications for experimental tests.

Abstract: In recent years, many researchers have proposed new models for synaptic plasticity in the brain based on principles of machine learning. The central motivation has been the development of learning algorithms that are able to learn difficult tasks while qualifying as "biologically plausible". However, the concept of a biologically plausible learning algorithm is only heuristically defined as an algorithm that is potentially implementable by biological neural networks. Further, claims that neural circuits could implement any given algorithm typically rest on an amorphous concept of "locality" (both in space and time). As a result, it is unclear what many proposed local learning algorithms actually predict biologically, and which of these are consequently good candidates for experimental investigation. Here, we address this lack of clarity by proposing formal and operational definitions of locality. Specifically, we define different classes of locality, each of which makes clear what quantities cannot be included in a learning rule if an algorithm is to qualify as local with respect to a given (biological) constraint. We subsequently use this framework to distill testable predictions from various classes of biologically plausible synaptic plasticity models that are robust to arbitrary choices about neural network architecture. Therefore, our framework can be used to guide claims of biological plausibility and to identify potential means of experimentally falsifying a proposed learning algorithm for the brain.

Formalizing locality for normative synaptic plasticity models
Colin Bredenberg"
Ezekiel Williams Mila- Quebec AI Institute Montreal, Quebec H2S 3H1
Mila- Quebec AI Institute Montreal, Quebec H2S 3H1
Cristina Savin
Blake Richards
Guillaume Lajoie Université de Montréal Mila- Quebec Al Institute Montreal, Quebec H2S 3H1
New York University New York, NY 10003
McGill University
Mila- Quebec AI Institute Montreal, Quebec H2S 3H1
Abstract
In recent years, many researchers have proposed new models for synaptic plasticity in the brain based on principles of machine learning. The central motivation has been the development of learning algorithms that are able to learn difficult tasks while qualifying as "biologically plausible". However, the concept of biologically plausible learning algorithm is only heuristically defined as an algorithm that is potentially implementable by biological neural networks. Further, claims that neural circuits could implement any given algorithm typically rest on amorphous concept of "locality" (both in space and time). As a result, it is unclear what many proposed local learning algorithms actually predict biologically, and which of these are consequently good candidates for experimental investigation. Here, we address this lack of clarity by proposing formal and operational definitions of locality. Specifically, we define different classes of locality, each of which makes clear what quantities cannot be included in learning rule if an algorithm is qualify as local with respect to a given (biological) constraint. We subsequently use this framework to distill testable predictions from various classes of biologically plausible synaptic plasticity models that are robust to arbitrary choices about neural network architecture. Therefore, our framework biological plausibility and to identify potential means of experimentally falsifying a proposed learning algorithm for the brain.
be used to guide claims of
Introduction
Over the last several decades, computational neuroscience researchers have proposed a variety of "biologically plausible" models of synaptic plasticity that seek to provide normative accounts of variety of learning processes in the brain-these -thee models aim to explain how modifications of cellular properties such as excitability and synaptic strengths can improve performance on a variety of important tasks for an organism, using only the information "locally" available a real neuron in the brain. However, there is no consensus on what "biologically plausible" really means. This is due to lack any precise definition of locality, with individual studies adopting different, architecture- specific heuristics that are often not explicitly articulated; further, an experimental level, the precise quantities available to neurons for computing synaptic updates are still up for debate [1, 2], and likely vary across cell types [3, 4, 5] and brain regions [6. 7, 8, 9, 10].
Corresponding author, email at: colin. bwwdenberg@mila.c..m quebec. Co-senior authors.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

As an example, some studies postulate that global neuromodulatory signals containing reward information could be available at individual synapses. in addition the more traditional pre- and post-synaptic information rypically required for Hebbian plasticity [11, 12, 11. Within this broad class, individual studies differ in their assumptions about this reward information (e.g. is it completely global [13] or potentially distinct for different layers regions [14]) and in the precise details about the pre- and post-synaptic signals available (e.g. is it spike-timing [15] that is available or spike-rate [16]?). Furthermore, there are many more signals that could theoretically be available at synapses: for example, other studies propose that individual pyramidal neurons could receive detailed error (or target) feedback information at their apical dendrites [17, 18, 19, 2] which can be used for learning- Previous work has taxonomized different normative plasticity models according to these feedback signals [20]. Within this space of normative plasticity models, it can be difficult to identify which features of given model constitute necessary, testable predictions that can be used for experimental verification-iiee if this information were not available a synapse, learning with the given algorithm would be impossible-ane which features are due to arbitrary choices that may vary with more or less realistic neural network architectures.
As a consequence these heterogeneous conceptions of locality and given the advancements in the field, it is arguably necessary to develop clear and precise way to identify what variables synapses should have access to in a learning algorithm, whenever claims of locality are made. In this paper, we focus on formalizing this process. Our central contributions are as follows:
1. We develop an architecture- independent formal framework for locality that term Sp- locality, which requires one to precisely specify the set of variables that synapses are assumed have access to within model: different choices of allowed variables for the set S produce different classes of locality. Importantly, this definition can be used for any learning algorithm applied to any stochastic neural network.
mechitectre-independent
2. We use our framework to group existing plasticity models into different locality classes. Intriguingly, we find that many different algorithms derive their locality from similar principles, even though they belong to different locality classes.
3. We show that our different locality classes make different experimental predictions, which makes it possible to identify which algorithms can be cleanly distinguished experimentally and which cannot.
2 Formalizing notions of locality
Our ultimate goal is formal definition of locality that makes it easy to identify which variables are locally available to synapses in a learning algorithm within any specific model architecture. However, this can be challenging. For example, when we consider kinlle-omppatmenn neuron model it seems reasonable to assume that the postsynaptic voltage locally available to synapses. In contrast, in a multi-compartment model, voltages can be different in each compartment, and synapses likely only have access to voltages within nearby compartments. Ideally, we would avoid a definition that simply requires us to list all of the variables that must be made available to the synapse, as that would carry little conceptual weight. However, we will begin by taking this approach, as it will help us to build the groundwork for a more conceptually useful definition in what follows.
Notation. We have X as random vector in R denoting network variables (e.g. voltage, spike- rate, inputs, etc.), and O as a random vector in Ne denoting network parameters (e.g. synaptic weights, spike threshold, etc.). We combine these as = [X, O] the concatenation of the two vectors. We will use Z p(Z) to denote a random vector with support Ω S R No+NNx but will also overload this notation to indicate a vector subject to either universal or existential quantification (e.g. 3Z s.t. df(2))
dZ = 0) or a dummy variable in an expectation (e.g. E,ZZ) [Z] = p)). We will use the subscript " i" to denote removal of the 7th element of vector, for example Z = IZ,...., Zi-1, Zi+,...., ZNx+Nel NNolT, and will occasionally use the more general notation Z where v set of indices, to denote the vector with the associated set of elements removed.
2.1 S-locality: graph-specific locality based on predefined set of available variables
We begin with a definition of locality that formalizes the concept of a set of variables, S₁ C Z, to which the th synapse can have access in order compute synaptic updates (e.g. some neural voltages.

---

reward signals, etc.). We then consider the set of these sets across all synapses, S = ) This inspires the following definition:
3. Definition 2.1. Given parameters ② E RR random variables X p E B and
function f() No + RNe, the update function f(Z) S-local with respect to some set S = [Sk Sx Z) and only ik
(Z) Oz,
3 E 1 s.t.
#0= 2, E S..
(1)
In short, S-locality requires that if parameter update fx (Z) has a direct dependence on Z₁ then Z; must be included in the set S2 of "allowed" variables for that parameter. For example, in the classic approach to locality in a neural network, for any synapse Waj E θ, we might set S = [r,,ry] S Z. allowing any S-local parameter update under this definition use pre- (r;) and post-synaptic (r;) firing rate information, and nothing else. If we wanted to allow a third factor such as a global neuromodulatory signal, R, to project to diffusely to every synapse in the network, as in reward- modulated synaptic plasticity rules, we might take S = (r,, r R). In this manner, for any model architecture, we can manually construct set Sij that defines our notion of locality for each synapse.
While our definition intuitive and flexible, it is cumbersome: for every parameter we have to explicitly define which variables are local. This means that S-locality is architecture-speciine requiring an evaluation of which Sy is an acceptable set of variables for each synapse's 's update rule. For example, if we switched from single- to nulti-ccomartmenn model, we would now have to modify S.j to accept only the voltage of the compartment where the synapse is located. Ideally, our definition would make obvious whether learning rule is local in both models without having to redefine the list of which variables are local. To do that, we will use the structure of the distribution p(X, θ) to construct definition of locality that captures intuitions across network architectures.
2.2 p-locality
Despite its ability to concretely define locality for any model, S-locality is architecture-specffi and cumbersome: in this section, we will develop an alternative operationalized notion of locality, called p-locality (where p stands for "probability"). Our idea is to formulate locality in terms of adjacency in the computational graph underlying network dynamics. To do this we will leverage conditional dependencies in the joint distribution p(X, θ) to generate an architecture-generi concept of locality'. This is considerably more useful both conceptual tool and organizing principle for normative synaptic plasticity models. However, we will still call on S-locality later to formulate our final locality definition: Sp-locality.
Further, it is important to note that for this definition, as with S-locality, we do not restrict the functional form of parameter updates fx(-)-we only restrict which variables they are allowed to depend on. This in contrast to previous, complementary work which has focused on defining allowable local computations and memory complexity constraints for 'biologically plausible' learning algorithms [21]. Our decision not to restrict fi(.)) enables us to abstract away the details of 'allowed' subcellular computations, and will consequently enable us to make much more general statements about the locality properties of normative plasticity algorithms as a whole, without having to focus on the specifics of particular models of neural dynamics.
Definition 2.2. p-locality: Consider a probability distribution (Z)) over network activity variables and parameters, and an update function f Nx +Na Re,, mapping variables and parameters to parameter updates. Assume that 0p(7. (Z.zz) and (lv(Z)
87. exist for all Z E Q and i, k indices. The
update function (Z) is local with respect to (p-local) if and only ik, i:
(f
#0= Ep (
(Ologp( (Z. d0
)
27
3 ZZ E S.t.t.
# 0.
(2)
OZ,
To unpack this definition and render it more intuitive, as with S-locality, see whether the kth index of f has direct dependence on particular variable Z, Z,wwe we assume
a/.(Z) indicates a test to 87,
'We define dynamics in terms of probability distributions because many canonical normative plasticity models stochastie; deterministic computation graphs are limit case of our approach.
3

---

a
oo
t
2
c
To
W
W
1
2
W
Figure I: Common neural network graphical models and the permissible parameter updates for a synapse Wi under p-locality. a. Feedforward neural networks can be described as DAGs, so variables that are not children or coparents of W ij are excluded from parameter updates under p-locality (by Property 2.1). b. For fixed weights a dynamic neural network like a recurrent neural network (RNN), p-locality excludes variables that are not children and coparents of Wi ij for all time points (also by Property 2.1). e. For an undirected graphical model like Boltzmann machine, variables that are not neighbors of W 'j are excluded under p-locality (by Property 2.2).
that fx(+) differentiable primarily for simplicity, and differentiable approximations can always be substituted in practice (-) happens to not be differentiable. If there a dependency, then we want that associated random variable Z; to be considered local to O g.- To measure this, have selected the Fisher Information of pZZ, [ee) with respect to Ø₁ This measurement quantifies direct influence. For example, Z; neuron downstream of neuron Zj. and O₂ is a synapse onto Zy, then by conditioning on Zj. O₁ is no longer able to have any effect on the conditional probability distribution of Z; through its influence on Z, In other words, the statistical influence of O₂ on Z; is mediated through the influence of Zj on Z; and thus it is indirect. By conditioning on Zj. we seal off one pathway of indirect influence, By conditioning on all Z we seal off all pathways of indirect influence, which makes
Blogp(2. Zad) 0 for all Z.
00
It is worth stressing that pZZ) = p>(( )( ()) is a joint distribution over both X and O even though most algorithms operate only on the conditional distribution p p(|/O). As consequence, the marginal probability that place on p(O) can be free choice, which is useful for clarifying the assumptions in the model. For example, we will typically assume that synapses do not have any marginalized dependence on one another, i.e. p()) = II p(O)).
2.3 Properties of p-locality
Having defined p-locality, we can now talk about its interesting properties, which will give intuitions for how it functions and which will figure prominently in our proofs of algorithms' locality. While any p-local update could also technically be reformulated as a S-local rule, by taking the set be the set allowed under p-ccallity, the conciseness of p-locality and its convenient mathematical properties will prove be its principal benefits over S-locality. For proofs of these properties, see Appendices A-C. To see the power and generality of p-locality as a definition, we first provide two properties that show how one can immediately make statements about the locality of a learning algorithm by simply inspecting graphs associated with the model. Assume all quantities are defined as in Definition 2.2 and that, for certain properties, the joint density satisfies mild regularity constraints (see Appendix C.I). For the first two properties further assume that (ZZ) is strictly positive (see Appendix B). Then we have the following:
Property 2.1. Assume Ga is Directed Acyelic Graph (DAG) for p. If Of2(Z)
+ 0 for Z; that is not
87,
parent, co co-parent, or child of O in Ga. then f is not p-local.
Property 2.2. Assume that G defines an Undirected Graph (UG) for p. If Ol12)) not neighbour of O₁ in G, then f not p-local.
82. # 0ffor Z; that is
By these properties, as long as the conditional dependencies of pZZ) can be summarized by an UG or DAG (two classes which subsume many modern neural network architectures, see Figure I for

---

examples), we can identify many of the variables disallowed by p-locality. Therefore, for such network, we can decide whether p-locality conforms to intuitions about biological plausibility as easily as we can for a hand-crafted set of allowed variables under S-locality; for most practical neural network architectures, we will see below that p-locality does behave intuitively. Thus, with p-locality we have definition of locality that does not require exhaustive list of the variables available to synapse, but rather, which relies on the implicit computational structure of the model (i.e. p(Z)).
Property 2.3. For any function b: Rx + Ne + JR defined such that by (Z) h2 (fa gu (Z)), where (Z) and (Z) are p-local and h2 is differentiable, ((z) is also p-local.
This demonstrates that p-local functions can be arbitrarily combined without the combination losing the p-local property. It also shows that p-locality places restrictions the functional form of parameter updates, so long as they are exclusively functions of variables allowed under p-locality.
Property 2.4. For any function f(.): : N Ne there exists probability distribution p(Z) such that the random variable fZ) with Z ~ p(Z) is p-local.
This is intended as a cautionary note: unless p defines a neural network-like probability distribution, y-locality does not necessarily conform to intuitions about biological locality (see Appendix F for examples). That (Z) is p-local for some probability distribution says nothing about function (ZZ; p-locality only becomes informative when narrow our focus probability distributions with biological relevance.
The following properties will turn out to be extremely important in Section 3; all algorithms that we survey below with provable p-locality properties will involve the score function or the derivative of the unnormalized energy function.
Property 2.5. The derivative of the log joint distribution alog ((X.e))
is p-local.
00
Property 2.6. For probability distribution given by p(Z) 1 exp (-E()), where Z normalizing constant, the expression 8 E(Z) is p-local.
Property 2.7. If the parameter marginal distribution factorizes as p()) Ih p(O)), i.e. the parameters are independent from one another. then the score function alogp is p-local.
Property 2.8. For mixture distribution P12
(Z 7) P1 (Z) (Z)'-7p(7) for some binary variable {0, 1} with nonzero probabilities, if (Z) is plocalll (or equivalently p2-local), then f(Z) is P12-local.
Intuitively, this feature holds because mixture distributions introduce more dependencies between pa- rameters and random variables X. Therefore, p- p-caaality under a mixture of probability distributions is more permissive than p-locality under any single one of its constituent probability distributions.
2.4
A simple motivating example
To make our definition p-locality more concrete, we will now explore how it functions for a common network architecture. Consider a simplified linear-nonlinear feedforward neuron model with L layers
of neurons (()) (X = [r(1) .... r(42)) connected by synaptic weights w (e W1 .... W())),
with additive Gaussian noise each layer. Conditioned on our feedforward weight matrices and stimuli s, the probability distribution of neural firing rates is given by:
p(rls, w)
III p(r 4 (111), w())
(3)
1=1 i=
p(r w()) N
((-1)),,²) 0²),
(4)
where for notational simplicity we have taken r (0) s, h(()) is pointwise nonlinearity, and w corresponds to the ith of W As discussed above, most algorithms operate on the conditional distribution, p(X p({(O), so we only define a prior over parameters for the purposes of assessing p- locality. To ensure that our joint distribution is directed graphical model, and to ensure that have no dependencies between parameters, we will assume that the parameters for the network are all independently distributed, i.c. p(W) HI p( (W!!)) where w is the synapse between
5

---

a
W
W
W
R
R
Figure 2: Different versions of Sp-locality, Rp-locality functions as p-locality, but also allows synapses to use rewards R for parameter updates, corresponding to an additional diffuse neuromodula- tory signal. b. "Ppmd-locality allows synapses use Pm Poclo variables and pa-local variables, as well as the global gating variable For the Wake-Sleep algorithm, a 7Pmad-loca probability distribution could correspond a network of pyramidal neurons receiving Pm -relate synapses their apical compartments (orange) and pa-related synapses in their basal compartments. The variable controls whether Pa or Pm. synapses affect cellular dynamics. Backpropagation and approximations are exp-local, which functions like p-locality, except it additionally permits a neuron-specific error signal to be used for parameter updates. Typically, e is constructed by sequentially propagating error signals backwards through feedforward network.
postsynaptic neuron i and presynaptic neuron j in layer 1. To make choice of conditional probability distribution more concrete (see Figure la), we can see that neural firing rates for this model can be sampled by:
(1) h(w()((-))) ())
(5)
where (7() ~ (0, 1). This corresponds to an ordinary multilayer perceptron neural network with noise added every layer. We can this model get intuition for how this choice of probability distribution p constrains the variables allowed for an update for a particular parameter w.O
Now, our probability distribution corresponds to directed graphical model with dependencies given by Eq. 3, so by Property 2, we know that variables that are not parents, coparents, or children of w violate p-locality if included in the parameter update for w!!) Figure summarizes the variables allowed under p-locality for such a DAG. w!!) has no parents under our graphical model, leaving only children and coparents. w!!)) has only one child, r: whose coparents are given by w(!) sk s.t. k + j (all other synapses onto r (1)), and (!-1) (all presynaptic neurons).
This shows that updates for synapses in simple feedforward network cannot depend neurons that are in layers > / or < ( - 1 while maintaining p-locality. We also found that dependencies on neurons within the postsynaptic layer | for indices k ب were also not permissible. As concrete example, by Property 2.7, we found that the score function of p(X ()) is p-local the parameters are assumed to be independently distributed, Our example satisfies this criterion. Its score function is
(r (T!"-((v!)))
h((V!"") where v w!ll ((--), By inspection, this function only requires information about the post- and presynaptic neurons, as well as the summed input to the postsynaptic neuron, V,.
given by: 8 log p(rls.)) aw!!
This simple example clearly outlines why p-locality aligns with expected notions of locality. More importantly, the use of p-locality also has the benefit of generalizing cleanly to alternative network architectures. For example, we were to instead inspect a multilayer recurrent neural network (Figure
1b), we would see that for (! the firing rates r!") (L) and all presynaptic neural firing rates would be permissible for all time steps, because the activities at each timestep are children or coparents of (I) For another classical example biologically plausible probability distribution corresponding to an UG (as in Figure Ic.), see Appendix D for discussion of linear continuous Boltzmann machine.

---

2.5 Sp-locality
In almost all situations, a given class of plasticity algorithms will produce updates that are almost p-local, in that synapses required to have access some small number of global state variables that a biological system could in principle project diffusely throughout a network, but which would not be allowed under generie p-locality (c.g. reward, which is usually produced far downstream of any given synapse in a neural network). For this eventuality, we introduce a notion called Sp-locality, which is best considered hybrid of S-locality and p-locality.
Definition 2.3. Given parameters θ Te- and random variables X RNx concatenated as Z = [ θ] and a function f(Z) NR-x*+Ne + IRNe, the update function f(Z) is Sp-local with respect probability distribution p(Z) and some set S (Sx Sa C 2 if and only if Vk fu = h4 (Sk, g (Z)), where h2 is an arbitrary function and gb (Z) is p-local.
The value of this definition is that it negotiates a compromise between the teahitectuur-generall of p- p-locality, and the flexibility of S-locality. Obviously if we take S₁ = Z Vk. all functions are S-local, and likewise, if we define p sufficiently generally, all functions are p-local (Property 2.4). However, if some class of optimization algorithms operating on predefined probability distribution p provably guaranteed to produce parameter updates that are Sp-local for some small set S, then we will have obtained very concise description of the types of information required by that algorithm.
3 Applying Sp-locality to normative plasticity models
Given a probability distribution over a neural network states and parameters, p(X, O), which could be experimentally motivated or even observed, it is natural ask which learning algorithms respect locality principles in the way parameter updates are made during learning. Alternatively, if we denote an algorithm which operates on a probability distribution p(X, θ) output an update fZZ) by A(p(X, ))) = fZZ) where f(Z): IR IR-Nx Ne - IR Ne, we can ask which probability distributions p(X, θ) and variable collections leave (Z) Sp-local. This is one way Sp-locality could potentially facilitate experimentally testing biologically plausible learning models.
To demonstrate the utility of our approach, in this section characterize the Sp-locality properties of a wide variety of algorithms that have historically been used to produce normative models of synaptic plasticity, summarized in Table 1. Though our framework covers many algorithms,, here we expand on three prototypical algorithms in detail, focusing on features that are relevant for our discussion of the algorithms' locality properties. More algorithmic details as well theorem proofs are provided in Appendix E. Schematics depicting biological interpretations for the types of Sp-locality discussed below are provided in Figure 2.
3.1 REINFORCE
REINFORCE [13], also known as policy gradient learning, produces reward-modulated Hebbian
parameter updates for neural networks [1] similar to the one discussed in Section 2.4. Here we will show that REINFORCE Rp-local, meaning that it is Sp-local where S (S,. = R Vk]. which assumes each synapse has access to a global scalar reward signal It that needs to be delivered diffusely to all synapses a network.
Theorem 3.1. If (()) = P(O)), the REINFORCE estimator given by An(p(R, is Rp-local.
As an example, consider the network defined by Eq. 3. The REINFORCE update for this network, for a single sample of the network state and reward R, given by:
R (1 n(V!"))
Dog p(rss W) ow!
(1 8 R. AWO
(11)
(6)
where again vi( - This is just the scalar multiplication of the reward signal R with the score function; because the score function is pocaal,, the full update Ry-local (see Appendix E for more detail).
7

---

3.2 Wake-Sleep (WS)
Whereas REINFORCE has been used to model reinforcement learning in neural networks, the Wake- Sleep algorithm has been used to model unsupervised learning [22]. It assumes that a neural network has two modes of operation, controlled by a scalar variable y that determines whether the network is in inference mode ('wake', . 7 1) or in generative mode ("sleep,' y 0). In all, neural activity samples are drawn from the mixture distribution given by Pma(r, >/W, M) = Pa(r Pa(rr " (r|M))-))p())) where p(7) defines the probability of sampling from the wake phase (pa: d corresponds to 'data') or the sleep phase (pmi corresponds to 'model"), and W corresponds to the feedforward weights while M correspond to feedback weights. For conciseness, all subsequent sections we will denote the mixture distribution pi (Z)7 p2 (Z) p(>>) by Mir (pi (Z), (Z)). As an example, building on the feedforward network defined by Eq. 3 by adding a generative feedback pathway, we have neural dynamics given by:
(1) yhw (W + (1-r)ha (1 (1+))) + on)
(7)
where (-) corresponds to the 'wake' nonlinearity, and (-) corresponds to the 'sleep' nonlinearity. Under this formulation, Y gates, for all neurons, whether activity is driven by the feedforward or feedback pathways; there are several hypotheses for how this could be implemented at a neuronal level, including global neuromodulatory or inhibitory gating of the apical and basal dendrites of pyramidal neurons in the cortex [23] (Fig 2b).
The Wake-Sleep algorithm updates M to fit Pm as generative model of the network's input data, and updates W to fit pa to perform approximate inference with respect to that generative model, in a manner similar to variational autoencoders (VAE) [24, 25]; unlike VAE training with backpropagation, however, the Wake-Sla -leep algorithm is Pomdllo meaning S = {Sk = =y Ak} (see Appendix E for the proof).
Theorem 3.2. If ((), timator given by Aws px is Poadloocll, where Pmd
(I (II (O) the Wake-Sleep es-
Mix (XIO), (())) p(O) ((1).
As concrete example, the Wake-Sleep parameter update for w (1) from the network above is given by:
ig for single sample of the state
(r.0) New
2 lag W) (1 (1- ow!!
(1) 33 (1 ?) !w!
Ny (V!")r
(8)
where V w (1) (1-1); similar update holds for M. Similar to REINFORCE, this parameter update only the combination of scalar variable y and the score function, and consequently Pmmd local (see Appendix E for the proof). Interestingly, Wake-Sleep is not the only algorithm to obey this form of locality: several other normative plasticity models, including Boltzmann machine learning [26). equilibrium propagation [27], and impression learning [23] have essentially the same p-locality properties. However, some of these have additional requirements on the probability distribution that make them less biologically plausible. For example. Boltzmann machine learning is yomallocal only for distributions that can be captured with energy-based models (which typically require weight symmetry).
3.3 Backpropagation (BP) and its approximations
Here we provide a characterization of the locality of the backpropagation algorithm [28]. which as we will show using our definitions, is only local algorithm with biologically implausible assumptions [20]. This demonstration is primarily important because the Sp-locality properties of several biologically plausible backpropagation approximations, including feedback alignment [29], weight mirror [30], and Burstprop [31] satisfy a similar notion of Sp-locality, but using more biologically realistic assumptions about (see Table and Appendix E). Derivations for the backpropagation algorithm and its approximations require more stringent network assumptions (here we assume a feedforward multilayer perceptron network as in Section 2.4), because the algorithm itself is not well-defined for arbitrary probabilistic network architectures.
8

---

Optimization Alg. REINFORCE
Locality supported
Architectural restrictions
Rp-local
DAG
Maximum Likelihood Estimation
Pm-local
DAG
Generalized EM
Pm Pm-local
DAG
Predictive Coding
Pm-local
MAP (weight symmetry)
Wake-Sleep
PPma-loca
DAG
Impression learning
PPmd-
DAG
Contrastive Divergence
PPma-loca
K-Step EBM (weight symmetry)
Equilibrium Propagation
Pmad-local
EBM (weight symmetry)
WTA-STDP
Pm Pocllcal
WTA
Backpropagation
e
(1)
p-local
MLP (weight transport)
Feedback alignment
6
p-local
MLP
Weight mirror
p-local
MLP
Burstprop RTRL
e p-locca
MLP
eJ p-local
RNN
e-prop
elpllcall
RNN
RFLO
e,p-local
RNN
FOLLOW
p-local
error-RNN
Table 1: Summarizing Sp-locality for classical learning algorithms. First column: the optimization algorithm interest. Second column: the variant of Sp-locality that we have proven in Appendix E. Third column: the network architectures required for both the algorithm and our proofs work. DAG = Directed Acyelic graphical model, MAP = Maximum a posteriori gradient descent dynamics, WTA winner-take-all circuit, EBM Energy-based model, MLP Multilayer perceptron, RNN recurrent neural network, error-RNN = error-driven recurrent neural network. 'K-Step' refers to the small, finite number of sampling steps required, as opposed sampling from a steady-state distribution or calculating an equilibrium, which are more time-intensive. Weight symmetry indicates that symmetric recurrent connectivity is required for the original proposed algorithm. Weight transport indicates that symmetric weights are required exclusively for error propagation.
Theorem 3.3. (f((O)= = 1I p(O)) and (( is defined by Eq. 3, the BP update for w with loss L(X), given by Anr()(X(()))))))) ((X) is for feedback alignment, weight mirror, and Burstprop are e p-local, where e! is given by their respective gradient approximations.
(1) e; p-local, where e; aF,"
dL Similarly, the updates
This notion of e!") p-loccaity, where S S- - c - Vi,j,(), includes restrictive biological assumptions than the one provided by Rp-locality and "Pmad-occaity, because it relies heavily on the neuron-specife error e!") which must be then accounted for using physiological mechanisms. For our network architecture given by Eq. 3, the functional form of all backpropagation approximations listed Table is given by:
AW ( a. (V))
where é; is an alperich--specific approximation of dL dF. Similar to Pocallocal algorithms, recent work has suggested that this error signal could be propagated through networks by top-down error signal relayed to the apical dendrites of pyramidal neurons [31] (see Fig. 2c). Thus, these algorithms make specific biological predictions, driven by the need for e!!) to be accessible to each synapse.
4
Discussion
We have constructed a formal definition of locality, Sp-locality, that combines set- based and prob- abilistic graphical model-based formalisms (S-locality and p-locality, respectively). Our goal was standardizing and operationalizing notions of locality, and we also demonstrated the usefulness of these definitions in characterizing normative plasticity models. Notably, this approach enabled the identification of distinct classes of Sp-locality (Table 1) which subsume learning algorithms spanning

---

more than thirty years of research, from Boltzmann machines [26] to modern incarnations like equilib- rium propagation [27]. Importantly, our framework can be applied to any neural network architecture, incorporating networks that are recurrent [16, 32, 23, 27, 26, 331, spiking [34, 12], and multilayer [29, 22, 30] into a single framework. This is thanks, in large part, to the architecture-genraain of p-locality, which abstracts away specific details of network models, defining locality instead in terms of statistical dependencies- -this makes our framework much more powerful than simpler linguistic descriptions of locality. As we discuss in Appendix F, this abstraction comes at the cost of requiring researchers to specify a distribution, p. that is grounded in biological constraints. We note that for non-biological probability distributions the variables allowed to be "local" under p-locality can violate standard intuitions; however, for standard neural networks like those discussed in Section 2.4 and Appendix D, p-locality behaves as one would expect, with "local" learning rules being those that use pre- and postsynaptic activity.
Table 1 organizes existing normative plasticity models into distinct classes. The first locality class, Rp-locality, requires individual synapses to have access to reward information; this class encompasses many sward-modulated Hebbian plasticity models [1]. The second class, Plmmatocalyty, encompasses predictive coding and its variations [33, 35], and aligns with more traditionally Hebbian plasticity updates (often at the cost of unrealistic network architectures; see Table 1). The third class, PPmdd locality, requires neural networks to have two distinct modes of operation (e.g. the 'wake' and 'sleep' modes in the Wake- SSeep algorithm) gated by scalar global variable y. Thus, "Pma-locality requires synapses to have access to the value of y to compute updates, and subsumes many algorithms that require generative feedback pathways for learning [22, 233. The final class of algorithms are (!)) p- local for feedforward neural networks, where e! is neuron-specific approximate error signal, and includes feedback alignment [29], weight mirror [30], and Burstprop [31], as well temporal variants of these algorithms [36, 37]. These four categories Rp-locality, P(mm)-occality, "Pma-locality, and to our knowledge, almost all existing normative plasticity models.
Because these plasticity models are sometimes only constructed for specific network architectures (which may violate known facts about the brain), we also specify the broadest class of network architectures for which we have successfully derived a model's Sy-locality properties (proven in Appendix E). As noted in Appendix F, 'biological plausibility" requires both network architectures and parameter updates to be in line with existing experimental evidence, while Sp-locality provides formal framework for assessing only the latter of these two factors. We therefore stress that Sp-locality properties are only relevant for neuroscience if they can be determined for biologically plausible network architectures. As consequence, classes of network architectures that encompass a broad range of biologically plausible network models (such as DAGs) are desirable, whereas algorithms that are only viable for implausible architectures (e.g. those that imply weight symmetry or weight transport) are undesirable.
Interestingly, the distinct classes of locality also delineate distinct sets of experimentally testable predictions for plasticity. Critically, Sp-locality abstracts away details that are not important for testing predictions, and helps identify important features of learning algorithms. For example, the REINFORCE (Eq. 6) and Wake-Sleep (Eq. 8) algorithms are equivalent with respect to the 'p' in Sp-locality, which tells us that the set of allowed local variables under p-locality itself (e-g. pre- and post-synaptic information) are not the best targets for experimental testing between these algorithms. and instead the focus must be on the variables included in 'S' (a reward signal R versus a clamping variable y that switches the network between different modes of synaptic plasticity and neural activity). Thus, by identifying whether plasticity is modulated by reward, neuron-specific error teaching signals, transitions in network dynamics, or none of the above, one may narrow the space
PPmd' or Plcaltoc respectively. These distinct phenomena are necessary features of the learning algorithms in question for any neural network architecture used and are consequently good targets for experimental testing. In contrast, other features of parameter updates are more about specific architectural choices, e.g. switching from a base- network [16] to spiking [15), from feedforward [22] to recurrent [32], or single- to multi contaitommrrtmennt [38]. Thus, beyond clarifying and categorizing the locality properties of normative plasticity models, our hope is that our framework will also support experimental efforts to differentiate between such models, helping the field to focus in on the most important predictions to test experimentally.
of possible candidate plasticity models down to Rp-, (1)
10

---

5 Acknowledgments
BR was supported by NSERC (Discovery Grant: RGPIN-2020-05110 Discovery Accelerator Supplement: RGPAS RGPAS-2020-00031:; Arthur B. McDonald Fellowship: 566355-2022), CIFAR (Canada Al Chair; Learning in Machine and Brains Fellowship), the Canada Research Chair in Neural Computations and Interfacing. GL was supported by NSERC (Discovery Grant: RGPIN-20 8- 04821), CIFAR (Canada AI Chair), and a Canada Research Chair in Neural Computations and Interfacing. CB and GL acknowledge support from Simon's 's Collaboration on the Global Brain Pilot Award. EW was supported by an NSERC CGS-D scholarship. CS was supported by the National Science Foundation under NSF Award No. 1922658.
References
[1] Nicolas Frémaux and Wulfram Gerstner. Neuromodulated spike-timing three-factor learning rules. Frontiers in neural circuits, 9:85, 2016.
teteetiiiiinnineeeeeeeee dependent plasticity, and theory of
[2] Blake A Richards and Timothy P Lillicrap. Dendritic solutions to the credit assignment problem. Current opinion neurobiology, 54:28-36, 2019.
[3] Claudia Clopath, Tim P Vogels, Robert C Froemke, and Henning Sprekeler. Receptive field formation by interacting excitatory and inhibitory synaptic plasticity. BioRxiv, page 066589, 2016.
[4] Yuhan Helena Liu, Stephen Smith, Stefan Mihalas, Eric Shea-Brown, and Uygar Sümbül. Cell-type- specific neuromodulation guides synaptic credit assignment in spiking neural network. Proceedings of the National Academy of Sciences, (18(51))e21 1821118, 2021.
[5] Matt Udakis, Victor Pedrosa, Sophie EL Chamberlain, Claudia Clopath, and Jack R Mellor. Interneuron- specifie plasticity at parvalbumin and somatostatin inhibitory synapses onto CAI pyramidal neurons shapes hippocampal output. Nature communications, 1 11(1))4395 2020.
[6] Dimitar Kostadinov, Maxime Beau, Marta Blanco-Poze and Michael Häusser. Predictive and reactive reward signals conveyed by climbing fiber inputs to cerchellar Purkinje cells. Nature neuroscience, 22(6):50-992,, 2019.
[7] Aparna Suvrathan and Jennifer Raymond. Depressed by learning heterogeneity the plasticity rules at parallel fiber synapses onto Purkinje cells. The Cerebellum, 17:747 75547755,,50 2018.
[8] Wei Guo, Blaise Robert, and Daniel B Pollcy. The cholinergic basal forebrain links auditory stimuli with delayed reinforeement to support learning. Neuron, 103(6):1164 1177 2019.
José Fernando Maya Vetencourt, Ettore Tiraboschi, Maria Spolidoro, Fero Castrén, and Lamberto Maffei. Scrotonin triggers a transient epigenetic mechanism that reinstates adult visual cortex plasticity in rats. European Journal of Neuroscience, 33(1) 33(1):49-57, 2011.
[10] Katie Bittner, Aaron D Milstein, Christine Grienberger, Sandro Romani, and Jeffrey C Magee, Behavioral time scale synaptic plasticity underlies CA 1 place fields. Science, 357(6355):0033-00666 2017.
[11] R Fiete, Michale Fee, and Н Sebastian Seung. Model birdsong learning based on gradient estimation by dynamic perturbation of neural conductances. Journal of neurophysiolog, 9844)22038-2057, 2007.
[12] Robert Legenstein. Dejan Pecevski, and Wolfgang Maass. A learning theory for deward-modulated spike- plasticity with application to biofcedback. PLoS computational biology, 4(10)xc100018) 1000180,
2008.
[13] Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Reinforcement learning, pages 5-32, 1992.
[14] Roman Pogodin and Peter Latham. Kernelized information bottleneck leads to biologically plausible 3-factor Hebbian learning in deep networks. Advances in Neural Information Processing Systems, 33:7296- 7307, 2020.
[15] Nicolas Frémaux, Henning Sprekeler, and Wulfram Gersiner. Reinforcement learning using continuous time actor-critic framework with spiking neurons. PLoS computational biology, 904)=e003024, 2013.
[16] Thomas Miconi. Biologically plausible learning in recurrent neural networks reproduces neural dynamics observed during cognitive tasks. Elife, 6:e20899, 2017.

---

[17] Robert Urbanczik and Walter Senn. Learning by the dendritic prediction of somatic spiking. Neuron, 81(3):521-528, 2014.
[18] Jordan Guerguiev, Timothy P Lillicrap, and Blake A Richards. Towards deep learning with segregated dendrites. Elife, 6xe22901, 2017.
[19] João Sacramento, Rui Ponte Costa, Yoshua Bengio, and Walter Senn. Dendritic cortical microcircuits approximate the backpropagation algorithm. Advances in neural information processing systems, 31, 2018.
[20] Timothy P Lillicrap, Adam Santoro, Luke Marris, Colin J Akerman, and Geoffrey Hinton, Backpropagation and the brain. Nature Reviews Neuroscience, (166)3335,446 2020.
[21] Owen Marschall, Kyunghyun Cho, and Cristina Savin, A unified framework of online learning algorithmss for training recurrent neural networks. The Journal of Machine Learning Research, 21(1):5320-5353, 2020.
[22] Geoffrey E Hinton, Peter Dayan, Brendan Frey, and Radford M Neal. The "wake-sleep" algorithm for unsupervised neural networks. Science, 268(5214):1158) 1558-1111, 1995.
[23] Colin Bredenberg, Benjamin Lyo, Eero Simoncelli, and Cristina Savin. Impression learning: Online representation learning with synaptic plasticity. Advances Neural Information Processing Systems, 34:11717-11729, 2021.
[24] Diederik Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint 131.41131,.6114 2013.
[25] Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic backpropagation and ap- proximate inferenee in deep generative models. In International conference machine learning, pages 1278-1286. PMLR, 2014.
[26] David H Ackley, Geoffrey E. Hinton, and Terrence J Sejnowski. A learning algorithm for Boltzmann machines. Cognitive science. 91):147-169, 1985.
[27] Benjamin Scellier and Yoshua Bengio. Equilibrium propagation: Bridging the gap between energy-based models and backpropagation. Frontiers in computational neuroscience, 11:24, 2017.
[28] Yann LeCun, Bernhard Boser, John Denker, Donnie Henderson, Richard Howard, Wayne Hubbard, and Lawrence D Jackel. Backpropagation applicd to handwritten zip code recognition. Neural computation, (4)):411-55, 1989.
[29] Timothy P Lillicrap, Daniel Cownden, Douglas B Tweed, and Colin J Akerman. Random synaptic feedback weights support error backpropagation for deep learning. Nature communications, 7(1):3326,, 2016.
[30] Mohamed Akrout, Collin Wilson, Peter Humphreys. Timothy Lillicrap. and Douglas B Tweed. Deep learning without weight transport. Advances in neural information processing systems, 32, 2019.
[31] Alexandre Payeur, Jordan Guerguiev, Friedemann Zenke, Blake A Richards, and Richard Naud. Burst- dependent synaptic plasticity can coordinate learning in hierarchical circuits. Nature neuroscience, 24()):1000-1019, 2021.
[32] G Hinton, Peter Dayan, Ava To, and Radford Neal. The Helmholtz machine through time. In ICANN-95, pages 483-490, 1995.
[33] Rajesh PN Rao and Dana H Ballard. Predictive coding in the visual cortex: a functional interpretation of some extra-classae classcclasicaa receptive-fielu effects. Nature neuroscience, 2(1):9 87, 1999.
[34] Peter Dayan and Geoffrey E Hinton. Varieties of Helmholtz machine. Neural Networks, 98)) 1996.
[35] Beren Millidge, Anil Seth, and Christopher I. Buckley. Predictive coding: theoretical and experimental review. arXiv preprint arXiv:2107. 12979, 2021.
[36] James M Murray. Local online learning in recurrent networks with random feedback. Elife, 8:e43299. 2019.
[37] Guillaume Bellec, Franz Scherr, Anand Subramoney, Elias Hajek, Darjan Salaj. Robert Legenstein, and Wolfgang Maass. A solution to the learning dilemma for recurrent networks of spiking neurons. Nature communications, 11(1):3625, 2020.
12

---

[38] Panayiota Poirazi. Terrence Brannon, and Bartlen W Mel. Pyramidal neuron as two-layer neural network. Neuron, 37(6):989-999) 2003.
[39] Judea Pearl. Probabilistic reasoning in intelligent systems: networks of plausible inference. Morgan kaufimann, 1988.
[40] Steffen L Lauritzen. Elements of graphical models. Lectures from the XXXVIth International Probability Summer School in St-Flour, France. http://www. stats, ac. uk/steffen, 2011.
[41] Richard E. Neapolitan al. Learning bayesian networks, volume 38. Pearson Prentice Hall Upper Saddle River, 2004.
[42] Christopher M Bishop and Nasser M Nasrabadi. Pattern recognition and machine learning, volume 4. Springer, 2006.
[43] Sam Roweis and Zoubin Ghahramani. A unifying review of linear Gaussian models. Neural computation, 11(2):05-335, 1999.
[44] Arthur P Dempster, Nan M Laird, and Donald B Rubin. Maximum likelihood from incomplete data via the EM algorithm. Journal of the royal statistical society: series B (methodological), 39(1):1-22, 1977.
[45] Geoffrey J McLachlan and Thriyambakam Krishnan. The EM algorithm and extensions. John Wiley & Sons, 2007.
[46] Ezekiel Williams, Colin Bredenberg, and Guillaume Lajoie. Flexible phase dynamics for bio-plausible contrastive learning. In Proceedings of the 40th International Conference Machine Learning, volume 202 of Proceedings of Muchine Learning Research, pages 37042-37065. PMLR, 23-29 Jul 2023.
[47] Shiro Ikeda, Shun-ichi Amari, and Hiroyuki Nakahara. Convergence of the wake-sleep algorithm. Advances in nearal information processing systems, 11, 1998.
[48] Bernhard Nessler, Michael Pfeiffer, Lars Buesing, and Wolfgang Maass. Bayesian computation emerges in generic cortical microcircuits through spike-timing-depnnde plasticity. PLoS computational biology. (4)) 1003037, 2013.
[49] Ronald Williams and David Zipser. A learning algorithm for continually running fully recurrent neural networks. Neural computation, 12):2700-20, 1989.
[50] Aditya Gilra and Wulfram Gerstner. Predicting non-linear dynamics by stable local learning in a recurrent spiking neural network. Elife, 6:e28295, 2017.
[51] Thomas L O' D'Donohue, William R Millington, Gail Handelmann, Patricia Contreras, and Bibic M Chronwall, On the 50th anniversary of Dalc's 's law: multiple neurotransmitter neurons. Trends in Pharma- cological Sciences, 6:305-308, 1985.
[52] Kevin Ellis, Catherine Wong, Maxwell Nye, Mathias Sable-Meyer, Luc Cary, Lucas Morales, Luke Hewitt, Armando Solar-I.ezan -eaama, and Joshua B Tenenbaum. Dreamcoder: Growing generalizable, interpretable knowledge with wake-sleep bayesian program learning. arXiv preprint arXiv:2006.08833 2020.
[53] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing atari with deep reinforcement learning. arXiv preprint 132 2013.
13

---

A Defining Markov locality and relating it to p-locality
To gain intuition for how p-locality functions, will introduce another notion of locality, called Markov locality, which will use the language of Markov blankets. We will prove that under relatively relaxed conditions p-locality and Markov locality are equivalent. This will allow us to relate the notion of locality to various graph structures commonly used to represent probability distributions, and will be key step proving Properties 2.1 and 2.2.
We start by defining the Markov boundary. M(X, S), of random variable X contained in a set of random variables 5, as minimal set such that p(X(S) p(X]M((X The Markov boundary defines a minimal set variables such that, conditioned on these variables, conditioning on no additional random variables in S changes the probability of X [39]. Similarly, define the Markov blanket, M(X,5) S) for in as any set of variables such that conditioning on ( makes X conditionally independent from all other variables [39]. In this way, the Markov boundary is Markov blanket but not all blankets are boundaries.
Definition A.1. Markov locality: Given probability distribution p(Z) and function f R. NRNx+NN
R.e,, the update function f(Z) is Markov-local with respect to the distribution over Z if and only V k:
(Z) Öz. +0 Z, M(
= Z E s.t..
(A.1)
Markov locality requires that the set of variables used in the parameter update fx (Z) is a subset of Markov boundary of the parameter itself. A Markov boundary can be thought as the set of variables that 'locally' communicate with the parameter O thus providing a natural measure of locality.
Importantly, for Markov-locality to be of use, we would like the Markov boundaries of random variables in the model of interest to be unique. Without this requirement there will be ambiguity, for a given in terms of which updates are considered local and which are not. To guarantee this, we ask that the conditional independence relationships implied by p satisfy four properties, commonly referred to graphoid properties [39, 40J. A sufficient condition for these to hold is that the distribution have strictly positive density (see Appendix for more details B). With this, and some mild regularity assumptions, we can prove the following equivalence between Markov locality and p- p-locality:
Theorem A.1. Assume all quantities are in A.I, that the conditional independence relationships implied by p(Z) satisfy the four graphwid properties given in Section B, and that mild regularity assumptions are satisfied by the joint distribution (see Section C. Then Equation 2 holds if and only if Equation A.l also holds.
Proof. This proof relies Lemma A.I, proved below.
We wish prove Eq.
+ Eq. A.I. It suffices show the following:
( a
e @ (Z.
27
Ep
#0 = Z, E M(O Z)
(A.2)
Using the contrapositive for the left and right implications separately shows that Equation A.2 is equivalent
(Dlog (l, (Zz,
)
27
E,,
=
Z & M( Z),
(A.3)
00
which means that suffices to prove Equation A.3 for the proof. Observe that
Ölo p(Z.
)
27
= = + alog pl Z4,) 0
Ep
A Z E Ω
(A.4)
a
azv
which follows from the regularity assumptions. From here, the proof follows by Lemma A.I.
14

---

Lemma A.1. Let X, θ Z, and i, and p be defined as Theorem A.I. Then
a log pl (Z,1Z
0
VZEN
Z $ M(Zx. 2)
(A.5)
ZZR
Proof. First, observe that
log (ZZ]
Z E
0Zk
Öp( (2 0
A
Z 2
(A.6)
by the chain rule. By applying the fundamental theorem calculus to this derivative, which we can do by the assumption of differentiability on I, we find that p(Z, Zyz) is constant w.r.t. Zg on 52 so that
ZZZ = = p(Z, / p(Zz) Zzii4))dZZ
(A.7)
= - Zn ( = p(z, Zz((.))
(A.8)
where we have also used that a probability distribution integrates to 1, and that (Z ( will be equal to outside Ω. Because Z Ω is arbitrary, from the above Z; is independent of Zy given the other random variables Z. Using Lemma 1 (which we can do by assumption of the graphoid properties),
see that if loog plog,pZZ, (Z4i) -0 A Z Ω by Eq. A.6 A.8.
J2.
Z. & MMZ,,, Z). Conversely, if we start with the assumption that Z; M(Zz. Z), we immediately get Ze & (Z, Z), by Lemma B.I, and see that p (Z. (Zy)) must not be a function of Zμ for all Z: thus, the derivative w.r.t. Zg is equal to zero for Z E Ω. Applying Equation A.6 completes the proof.
B Notes on Probabilistic Graphical Models
In this section we compile several properties, definitions, and results on Markov boundaries and Probabilistic Graphical Models (PGMs) that underlie Theorem 1. and Properties 2.1 and 2.2. We begin by setting up notation. Let us assume that we have joint probability distribution, P. over set of random variables S, and that W, X, Y, Z S, and U. V. T E S. We X JL YIZ to mean that the set of random variables X is independent of the set Y given set Z, and assume that the reader familiar with the notion of directed graph, an undirected graph, and graph separation. If set X contains only single random variable U then we abuse notation and write U in place of X.
The following four properties-knowe the graphoid properties (or axioms-ses e.g. [39, (0)440))-a0 are useful for getting well-hehaved Markov boundaries and in assigning graphical representation to probability distributions:
Definition B.1. Pseudo-graphen properties:
Symmetry: X IL L|Z = Y 1 X|Z
Decomposition: X IL Y, Z = x TT YIZ & X T wIz
Weak union: X IL Y, Z = X IL YIW,2
Intersection: X 11 Y W & X I WIZYY = XLYWIZ
15

---

Importantly, it is known that these properties are satisfied when we have density p that is strictly positive w.r.t. to its base product measure [40]. Here, measure used measure theoretic sense; e.g. we have assumed throughout the paper that the hase measure is simply product of multi- dimensional Lebesgue measure over RR for some kj, and counting measure over N or some subset of N for some k2. Roughly speaking, this positivity property means that there are no purely deterministic relationships between variables.
The key result that we use to guarantee that the Markov boundaries we discuss in the paper are well-defined is given in [39]. We state a paraphrased and shortened version below for completeness:
Theorem B.1. Theorem 4, Chapter 3 in [39]: every P with conditional independence relations satisfying the four pseudo-graphoid properties has a unique Markov boundary for each X.
We now add two more simple results on Markov boundaries, used in the proof of Theorem A.1: Lemma B.1. If P has conditional independence relations satisfying the four pgeudo-graphoil properties we have:
for every U,V E S, E M)..S) = Ve M)U,S)
for every E S, M(U, S) contained in every Markov blanket of U.
These follow simply from the graphoid properties omit the proof.
Lastly, make specific what we mean when we say that a graph an undirected or directed graphical model for distribution.
Definition B.2. Let G be an undirected graph where each node corresponds to random variable in S. We say that G an Undirected Graph (UG) for whenever X and are separated by Z in G. X IL Z is true under P. Note that this corresponds to the notion of I-map in [39].
Definition B.3. Let Ga be directed graph with each vertex corresponding to random variable in S. We say that Ga is a Directed Graph for P the variable under corresponding any node the graph is conditionally independent of all variables corresponding to nodes that are non-descendants given the variables corresponding to parents. This equivalent Ga satisfying the Markov condition described in Definition 1.9 of [41].
C Proofs for p-locality properties
For the first two properties we assume the requirements of Theorem A.I are satisfied. For all properties except 2.3, 2.4, and 2.8 we assume satisfies mild regularity constraints (see Section C.I).
Property 2.1 Assume Ga is a Directed Acyelic Graph (DAG) for If 0fa(Z) parent, co-parent, or child of O₁ in Ga, then f is not p-local.
82. * 0 for Z; that is not
Proof. By Theorem A. we get that (Z) 02. can only be non-zero on the unique Markov boundary of Z, is p-local. By the definition of DAG, the parents, co-parents, and children of Z, form Markov blanket for it (see e.g. [41] Th. 2.13). By Lemma B. the boundary is included in all Markov blankets (Z) can only be non-zero some subset of the parents, co-parents, and children of 02,
Z.
Property 2.2 Assume that G defines Undirected Graph (UG) for p. If 0fa(2) a neighbour of O₂ G, then is not p-local.
+ 0 for Z₁ that is 82,
Proof. As above, by Theorem A. we get that (f. boundary of Z, it is p-local. UG for a distribution is an I-map for it, and conditioning on the neighbours in I-map renders node independent from the other nodes in the graph by definition- thus the neighbours form a Markov blanket. By Lemma B. the Markov boundary is included in every blanket so O2.
82. can only be non-zero on the unique Markov
(Z) can only be non-zero some subset of the neighbours in the UG.
D
16

---

Property 2.3 For any function b(z)
N 7 RN defined such that (Z)
(f)(((Z), (Z)), where f and g are p-locat and h2 differentiable, b(z) is also p-local.
Proof. Suppose that 8bg(2) ( ix;
# 0. We need to show that Bp
(
(olos Poggpt piZ, [aaze) D6,
* 0. Knowing that
(2) ax. + O, we have:
Shg (Z) (Z)
(Z) 0gk (Z)
0.
(C.1)
OZ;
Ogk
0x;
This implies that either (2) oz:
* 0 or
(g.
+ 0 (or both). No matter which true, by virtue the 82.
p-locality of f and 9. we have the consequence:
(Olog p (Z, Zzzi xx)
Bp
# 0,
(C.2)
ae
which concludes our proof. This demonstrates that p-local functions can more or less arbitrarily combined without the combination losing the p-local property.
Property 2.4 For any function f(-): : Rx R Re,, there exists a probability distribution p(Z)
such that the random variable fZZ with Z ~ p(Z) is p-local.
Proof. We can prove this property by construction. Take p af(x) 0)
0x:
ie. the distribution of every variable contained within has mean parameter dependence on all θ
variables. The probability distributions for all other variables are otherwise unconstrained. Then
for all such that 0f(x) +
O, we have:
2x,
( X gii 6
O (X,gp(X;1(0)
Ep
Ep
(C.3)
00
i
(X: 0)
Ep
(C.4)
2
Ep
2 ((X: - 0' 0))))))
(C.5)
(Ep())
(x
-x))
(C.6)
4Ep())
[0]] 40.
(C.7)
Property 2.5 The derivative of the log joint distribution Olggp(X
is p-local.
Here, it's more useful to work with the equivalent (contrapositive) requirement for p-locality, i.e., we need to show Vk, i:
Olog (0,gg(Z.22 (Zz) 20,
)
27
8 log (0m,pXXX000 (X θ) 0.
Ep
= =
(C.8)
82,0
Proof. First, that:
2 Olog ponpp(Z, Ologp(Z, Zzzii
27
Ep
=
(C.9)
00;
i log pl (ZZ, (4i)
0
(C.10)
00
8 log (Z. Z Z4s))
0,
(C.11)
oz,00k k
17

---

where the first implication follows from the fact that the Fisher Information integral is effectively weighted sum of elements, each of which Z, then the Fisher Information would also be nonzero. Assuming that log has differentiable partial derivatives, we can interchange the order of differentiation, giving:
0. the function on the right were nonzero for some
8 log p Z2 Zzzi
=0
(C.12)
0
[log p( [z
log p(z =0
(C.13)
8
Oz;
0
0
[log p (Z))
(C.14)
00 k
OZ;
8 log (Z)
0,
(C.15)
0Z
which concludes the proof.
Property 2.6 For probability distribution given by p(Z) = exp à (Z) is p-local.
exp (-E(Z)), the expression
86
Proof. The proof is almost identical to the proof for Property 2.5. From Property 2.5, we have that:
(
( Pog Zzzi)
27
Ep
(C.16)
H
8 log p(Z)
0.
(C.17)
OZ,00,
Using our definition p, we have:
0 (E(Z) + +og Z) 0
(C.18)
07,00k
8 EZ) 0,
(C.19)
OZ,00*
which concludes the proof.
Property 2.7 If the marginal parameter distribution factorizes p()) P(O)). i.e. the
parameters are independent from one another. then the score function a pl (9)
is p-local.
de
Proof. Again, we make heavy use of Property 2.5, which states:
(
O ( (Z. Zez ) 27
)
82 log pZ)) 0.
Ep
OZ,00, =0 -
(C.20)
It is important to note that the left-hand equation only holds true if Z; * O₁: under p-locality, an update equation for parameter O can always include its own value. So for the remainder of the proof we will assume that Z; * Oç. Now, log(p(X 0)) log p(Z) log p(e). We also have:
8 lo)
82 Σ. log p ( 0,
(C.21)
oZ.00 k
a *
where the last equality we have used the assumption that Z. + Oc. These two equations collectively imply:
8 (log)(((())
82 log (Z) log pl 4) = 0, az,00;
(C.22)
az,00;
which concludes the proof.
18

---

Property 2.8 For mixture distribution P12(Z. 7) =Pi (2))22222/0000) p(nn) f for some binary variable 7 {0, 1] with nonzero probabilities, f is pi-local (or equivalently pz-local), then f(Z)) is P12-local.
Proof. We again work with the contrapositive definition of p-locality, observing that:
27
(z,,))
= 0
(C.23)
i Olg pie Pi (Zz)
27
E
p(7 = k) Pp)
=0
(C.24)
d0p
AE{0,1}
① Olggp (Z, Zyyi)
27
= E (x,(x
(C.25)
09k
fa (X)
(C.26)
OZ.
where the third implication follows from the fact that if the sum of two nonnegative quantities is zero, then both quantities zero, and the final implication holds from the plllocality of fZZ).
C.1 Regularity of Joint distribution
For several of these properties enforce mild regularity constraints the density. This is because we want the integral of the squared score being equal to zero to imply that the score itself is equal zero. A sufficient condition for this is that the joint density function and partial derivatives w.r.t. O₂ vk are, for every fixed value of Z's discrete elements, continuous functions of Z's continuous elements.
D Locality for a linear continuous Boltzmann machine
Consider the following example of a simplified linear recurrent neuron model with synaptic weight matrices W. The joint distribution is given by:
1 -E(r.W)
p(r, W)
(D.1)
Z
E(r, W) =
|r|
T Wrr
+ 2 [[B3 /02
1
(D.2)
2r
2
Σ
r
Wor.r + 2 Σ w /02, 2
1
(D.3)
2T
where W is assumed to be symmetric matrix. To see why this probability distribution is relevant for neuroscience, we first note that E is linear, continuous analog of the Hopfield energy function, which also used for discrete-valued Boltzmann machines. There are two critical differences between this probability distribution and the linear feedforward network explored in Section 2.4: first, this distribution corresponds the undirected graphical model shown Figure 1c, as opposed the DAG shown in Figure la; second, the marginal distribution is not a free parameter that we can choose with convenient factorization properties we want our joint distribution give us a version of p-locality that corresponds with our concept of biological locality. For undirected graphical models, one typically is required to define the joint distribution first, and compute conditional distributions explicitly through Bayes theorem approximate through some form of MCMC sampling. In our case, we can see that p(r W) corresponds to the steady-state distribution stochastic differential equation using E to perform Langevin sampling:
19

---

dr; (1,,()) W) dt odB;(t)
(D.4)
-n(() + E Wirr(()) di + (I),
(D.5)
where here (t( corresponds to uncorrelated Brownian noise injected into the system. These stochastic sampling dynamics correspond to a noisy linear recurrent network. Therefore, p(r)W) corresponds to the steady-state stimulus response distribution of linear recurrent network.
Let's ask: for which neural indices & can we have
8rg a_ fw., (r) * 0 so that the function f is still
p-local? For f to remain p-local, we would need Ep ( ( (Ologpíre p(ra ( W]
2
* 0. For the definition
OW.y
of p-locality to conform to our intuitions about biological locality, we would expect the only allowable variables to be the pre- and postsynaptic neurons r; and r, we will show that including any other variable will violate p-locality. To see why, suppose k o i, Note that we can decompose E as:
E(r, W)
Ex + Expk
(D.6)
Ek
r
1
Σ
W WayTer Σ
W
Warrfrk 102
(D.7)
2T
2
j
Exp
Σ
?
W Wirry
1 Σ w 2
102.
(D.8)
Under this decomposition, E& has no dependency on Waj. and Eyx has no dependency on Fa. Now we're in position to demonstrate that for any choice of k such that k i.j. Sw., (r) cannol be p-local.
p(r W) p(r4K, W)
p W)
(D.9)
p(r[W) p(r, W)dre
(D.10)
E-E J
(D.11)
(Ex Ege)
(D.12)
-gn J (Ex)
(Faa)
(D.13)
f e (-(B)
logr(ra/rys,
W) =0
(D.14)
ow
27 i Oog Hog pr W)
Ep
O.
(D.15)
aW
Because the conditional distribution has no dependency on W i.. then the Fisher Information is also which concludes the demonstration: fw., is not p-local if a function of rą for * i, Of course, this decomposition of E = Et + Ezi would not be possible if our simple example, any parameter update for W,y that depends on the activity any neuron ra that is not the pre- or postsynaptic neuron (k: & i, cannot be p-local. Alternatively, since this is an undirected graphical model, we can also inspect its corresponding graph (summarized in Figure Ic.). To verify that the graph in Figure 1c. corresponds to network, observe that our probability
or j. To summarize, for
20

---

distribution factorizes according the cliques of the graph [42] as follows:
p(r, W) = 2
II Φ ra
Φ W
sy) II
or,, W.y)
(D.16)
17
o(r;) = exp
(D.17)
2TO²
1
o o(W;) exp
2
(D.18)
20²
o ry. W;)) exp
W (5ra,
(D.19)
2
Looking at the graph, verify by inspection that the only neighbors of Wej are r; and r which confirms our detailed analysis by Property 2.2.
E Proofs of p-locality properties of normative plasticity algorithms
E.1 REINFORCE
Theorem E.1. If p(O) II (())), the REINFORCE estimator given by AR(p(RR is Rp-local.
Proof. The REINFORCE derivation proceeds as follows: suppose that we have some probabilistic formulation neural network and incoming sensory stimuli p( ()) and some probabilistic reward function p( [ dependent on the stimuli and neural responses. We want to maximize expected reward:
E = [X(
(E.1)
we want modify our parameters @ in order to improve performance, we take steps in an approximation of the direction of the gradient of the objective E(R]
/
R
X (
(E.2)
d
8
/ Rp(R
a (XO))XXR
(E.3)
/
Rp(R[X)
logr(X|
dXdR
(E.4)
70
/ Rp
0
X
log
((X
(E.5)
d0
K
(k)
log p X (),
(E.6)
K
d0
Rm
where in this last step we have employed a Monte Carlo approximation of the expectation, where R(R) and (2) ~ PRR, X). This update function: fRR, Z) R
x 0. However,
s log p(X(e) is not p-local know, this update is the
à
because we have (R, Z)/OR o O, while p( product score function with a marginal parameter distribution that we have assumed factorizes, which know to be p-local by Property 2.7, with scalar reward R. In this case, could postulate that reward information is projected broadly to many synapses in the neural network via a neuromodulatory pathway (Figure 2a). We see that (R, X) hRR, (X)) if take h(a, axx b and gXX) 35 lo O)); further that y is p-local, and hence [RR, X) is by Definition 2.3 Ry-local.
8
21

---

This might seem contrived, because any function is Sp-local for some sufficiently broad choice of S. However, we have shown here that the REINFORCE algorithm is Rp-local for any choice of p with marginal parameter distribution that factorizes (an easy constraint to satisfy for directed graphical model architectures). This means that we can make any of a huge variety of neural network or probabilistic model choices and still have the REINFORCE algorithm obey the same notion of locality, without having to modify our definition post-hoc.
E.2
Maximum Likelihood Estimation (MLE)
MLE is a highly popular machine learning method and the fundamental basis for several subsequent normative plasticity algorithms. This algorithm involves fitting model, pm (X|@), empirical data distribution, Pa (X).
Theorem E.2. If (O) II ((O)), the MLE update given by AMLE(Pm (X|O), pa())) is pm
local.
Proof. We proceed by first deriving the MLE update function. The objective function for maximum likelihood estimation is given by the KL divergence between an empirical data distribution, Pa(X) and probabilistic model of the data pm (X We have:
Pm (X
(X Pm x = - log
Pa Pa(X)dX.
(E.7)
Px(
We want to minimize this objective function, which we do by gradient descent:
ae / log
Pm
px(X) (
AMLE(p((X) Pm (x)) x
Pd (X)xxX
(E.8)
. d
log (pm Pa()ddX
(E.9)
θ
log (pm (Xx
(E.10)
K
where X₁, ~ PX),,, and in the last approximate equality we have used a Monte Carlo sampling integral approximation. This update exclusively contains the score function of pm, so by Property 2.7, the update is Pm-local
E.3 Generalized EM (GEM)
MLE estimation runs into difficulties when attempting to fit latent variable models, (e.g. when Pm (X.) = Pm (X., X1.dXA,) where X,, are latent variables within the model distribution that "explain" observed data X Latent variable models are extraordinarily powerful, and appear in computational neuroscience in a variety of forms, including but not limited to factor analysis, hidden Markov models, and Kalman Filters [43]: for these models, we will take X [ X,). Instead of performing explicit MLE, when fitting latent variable models one usually resorts to some variant of the Expectation-Me -axiittiiio-aiiiiiiaiiiioooo (EM) algorithm [44]. Here, show that particular variant of the EM algorithm, called Generalized EM (GEM) [45], is -mcll in the same way as MI.E.
GEM gains computational benefits by substituting (by any of a variety of methods) an approximate posterior distribution (X( X.,) for the true, but typically intractable, model posterior Pm (X,, (X,) via minimizing variational free energy [45]. However, GEM is not just convenient model-fitting algorithm: in subsequent sections, we will show that the pmm Pmollicy of GEM explains why several popular normative plasticity algorithms produce biologically plausible updates.
Theorem E.3. If p()) = ((O)), the GEM update given by ( Px))) is Pm Pocllcal.
Proof. Rather than minimize KL[pa (X,) Ipm ()]. the GEM algorithm minimizes an upper bound (the variational free energy). Taking X [X., Xnl:
KL(pa((X)llpp. (X,, [)] 2 KL(p, pm (X, [)] + (x.) [KL [KL|pa(Xx, (X)) (X, X., O)]
KL(px(X) 1pm [X [)),
(E.11)
22

---

where the inequality follows from the positivity of the KL divergence. Here, Pa X) is an approximate inference distribution. Different choices of how this distribution is selected/optimizze can produce very different learning algorithms, with varying degrees of biological plausibility. Obviously, the loss is minimized with respect to Pa(X, X) if (X [ = Pm (X,(X, 2a) (where O = θ prior to optimization wrt )). This choice corresponds to GEM [45]. For now, we will not concern ourselves with how (a(X,, XA, X) is selected-instead we will focus on the locality properties of gradient updates of this loss with respect to m.
Having packaged hidden and observed variables together (X X X, our derivation proceeds exactly the same as for MLE:
AGEAt (X()) x
KL (LIpx(X)IImmXXI))
d0
/ log (m
Pa Pa(xX
00
Pa(X)
/
log (Pm((X()) Px
00
log (pm Xx, X(O)),
(E.
de
where with the MLE update, X5 Pm (X This update is pm-local for the same reason that the MLE update is.
E.4
Predictive Coding (PC)
As an additional note, if one takes the approximate posterior pa (Xx, X.) to be given by:
Pa x xx)
argmin KLIpa(X)llpm (X(O) s.1. pa (X, X.)) (X.).X))) (E.13)
Pa(X,
where 5(-) indicates a Dirac delta distribution and X, (X,) indicates a set of observation-depeeddent mean parameters, then recover the predictive coding family of algorithms [35]. Typically, in this
context for given observed stimulus X., Pa(X, [X,) ~ 6(x,(X.) )) estimated by reparameteriza- tion and gradient descent with respect X, (X,) (a mean parameter that is ineevvation-ependeene which-for clever choices of Pmo-lly resembles the dynamics of a recurrent neural network relaxing to a stimulus-conditioned equilibrium state [33]. After estimating X₁, (X.), parameters O are updated as in GEM. Therefore, the derivation above also applies to predictive coding algorithms, which are consequently also Pm -local.
Theorem E.4. If
1I (1..(OOg, p(O)), the PC update given by Arc (XIO), is -lcal.
E.5
Wake-Sleep
Unlike the previous three examples, which only require sampling from the pa distribution and only calculate parameter updates according to the pm distribution, the Wake-Sleep algorithm parameterizes both distributions and jointly samples from a mixture of the two distributions across its 'wake' and "sleep' phases. As we will see, this will mean that the Wake-Sleep algorithm will end being local, where 7 is the binary variable that controls whether the system is in its 'wake' or 'sleep' phase.
Theorem E.5. If p(O, O(d) timator given by Aws(P p) is "Pma-local, where Pnd Mix (pm. (x p O (d)).
(IT)((O)) (O. the Wake-Sleep es-
Proof. Our updates similar loss to the GEM algorithm, namely take:
Aws (X))
AO A0 (d
(E.14)
where AOws is given by:
23

---

0
A0ws x
KT. Pd X @ (1
(X(m)
d
K
2
log (pm (XA, X(O)).
(E.15)
K
d0
k=0
whereas for GEM, we sampled only from Pa- Because each term of this update is 0 "/ * 1, this update still an unbiased estimate of the gradient [46] and is effectively equivalent to the GEM update, except that it allows the system to alternate between sampling from Pm and Pa. This alternation is useful because will allow also calculating parameter updates for A For ((d), we optimize the reverse KL-divergence"; by directly analogous derivation to Eq. 12, the update is given by:
Here, 7h. X ~ Pmm(( ( (())..
AO
(d) Wg
X
d0
KL (X)))))(((((()))
2 K
7k) () log (p((X((())))
(E.16)
Now, these updates contain the score function for both Pm and Pa. as well as the scalar mixture variable 7, As a consequence, both updates "Ppma-local, via Properties 2.7 and 2.8 (slightly more precisely, AO is 7Pm local,, and A is yypa-local).
E.6 Impression Learning (IL)
The impression learning parameter update [23] is closely related the Wake-Sleep parameter update, and is consequently also "Pmal- What distinguishes IL from WS is the use of rapid alternations in the gating signal % within single trial with 7 time steps. Here, X [Xo: .... XT].
(X, 1)1 1 a mixture distribution in which y alternates between 0 and 1, sampling from either Pm. pa at the time step t, respectively.
= [70 .... Yel and Pma (X/%, ( y )) = II' pa (X,(X ²)
Theorem E.6. If p(e) = IL the impression learning estimator given by ( Px (X(O)) is local, where Pmd = Mix (xX))) p(O).
Proof. Similar WS, the update is given by:
/
a
A x
Σ
(X= + 7pm (X, X, 0)
Pmd X O)dX
-
(E.17)
T
a log ps (X + 71 g0 pm (X,
7t)
(E. 18)
0e
1=0
where in this last equality, X ~ Pmd PMd) and tion. It is worth noting that unlike in the Wake Sleep algorithm, here "/. is not a constant throughout time. Instead, "t alternates between and with 'phase duration' K, i.e. 7t+1 mod (t,, K) = 0, and 7 = 7t otherwise. The II update is the score function of Pmn(X 7, θ). which has identical dependencies the score function of Pmm(X 7, Q) (only change from + 1 1-11 has occurred). Therefore, if ((O) = II ())), this parameter update is 7ppma-local by Property 2.7.
performing single-sample gradient approxima-
7t. if
'A rigorous discussion of why this optimization process is sensible is beyond the scope of this manuscript.
See [22, 47] for more detail.
24

---

E.7 Contrastive Divergence for Boltzmann machines (CD)
While the GEM learning update is provably Pm -local, is also predicated on the assumption that the parameter marginal distribution factorizes as pl ()) III pl (x),, which as we note in Appendix D can be difficult to ensure for even simple undirected graphical models. An extension of the GEM algorithm, we can show that the CD algorithm is 7/mmallocal (as opposed to just pm -local) under less restrictive assumptions. The cost of this is that CD learning usually requires costly MCMC sampling from both the posterior distribution pm (X,, Xx,, ②) and the full joint distribution pm (Xi: ())
Pm.(X..
Theorem E.7. The CD update given by ACD AM (X()), is "(Pmd-local, where Pmdd Mix (pm (x(O), (X()) (()).
As mentioned above, for GEM the most natural choice for pa ma(X,, is given by Pm (X XX, O As we demonstrated, parameter updates calculated according to this rule will be "/Pma-local, but there two important caveats. First, for GEM produce biologically plausible updates, we still need biological system that can sample from Pm (Xx XX 6o). Second, it is important to remember that we are only guaranteed that the score function is guaranteed to be p-local if the marginal parameter probability distribution factorizes as p( (() II p( (x). ). For DAG, it may be difficult to satisfy the first condition without approximation (given by the Pake-Sleep algorithm, for instance), whereas for an UG, it may be difficult to satisfy the second condition, as we saw in Section D. To make our update "PPmd -loall for an undirected graphical model like the Boltzmann machine, we will require an extra step that we outline here to use the energy function Property 2.6 rather than the score function Property 2.7.
Having committed to working with an undirected graphical model, instead of sticking to the original GEM update, here we break apart the probability distribution as:
ACD(Pm (1)0)((())) = / log (pm Px Pa (X)
(E.19)
20
[log (E(X, +)) log 2 2(H)) (X)
(E.20)
/
E(X, O)p((aX
log 2
(E.21)
00
10
ʃ 00
à E(X,O))a(X)))
0
E(X,0) of
dX
(E.22)
3(0)
00
/ 00
0
/
0
E(X. (X) +
E(X, θ) E ,) dX
3(0)
00
(F.23)
/
0
(X)xX /
0
F(
F(X, F((,,)) Pm.
(X/O)xX
30
(E.24)
2 K
0 00
11
E(X,, θ),
(E.25)
where X₃ is sampled from Pmd- Now, by Property 2.6 and Property 2.8, this update local- This is the Boltzmann machine learning algorithm [26]. where the clamped and unclamped phases are alternated between stochastically [46]; for our previous linear Boltzmann example, in which X and θ = W, the derivative of the energy function (Eq. D.3) with respect to parameter W:; is r.ry. demonstrating that updates correspond data distribution given by Pa(r,) for some r, S and Hebbian updates are positive, contrasted with an unclamped phase in which updates are negative. Note that for this model approximate sampling from a posterior distribution Pm (ryoorr Ir. W (a))(ro) is no more difficult than sampling from the joint distribution: one simply holds r. fixed to an environmental data sample and performs Langevin sampling on all other variables.
two phases of updates: one in which r, clamped to a
25

---

E,8
Equilibrium Propagation (EP)
Though the derivation and setup of the equilibrium propagation algorithm [27] is very different from Contrastive Divergence, the functional form of the derived update is very similar. While equilibrium propagation typically operates on deterministic networks, here we will provide our derivation for the stochastic version with an energy function defining joint distribution over θ and X (as in Section D), which is somewhat more straightforward to into the p-locality framework.
Suppose that we have a probabilistic energy-based model whose energy function is given by:
E e) (Z) + eL(Z),
(E.26)
where e E 8), where 3 < 1, where ((Z) is the loss function that the parameter updates are optimizing. This be thought of "soft-clamped" system, in which nonzero pushes the system towards slightly better performance. Intuitively, the EP parameter update attempts to change network dynamics so that the unclamped system is nudged towards the slightly better performing soft-clamped system. Then we have the following theorem:
Theorem E.8. The EP update given by A 7) is PPmd-local where Pmd Mix (p(X(O, = p() 0)) p(O).
The parameter update for equilibrium propagation is given by:
dE c)
(E(Z,e)
A &
E Ex=
-E=00
(B.27)
B
00
K
11) (L Eb.
2
(E.28)
00
k=
where for the final equality we using campling-based approximation in which are sampling from X ~ p(X]O,<)) and c\/3 3 ~ Bernoulli(O.5)) This is almost identical to the Contrastive Divergence update, except that rather than clamping neural activities slightly biased towards better performance. Because this is the combination of the derivative of the energy function with a mixture variable E BB, by Properties 2.6 and 2.8, this update is ((O).
target output, they
Poma-local where Pmd = Mix (x(X|e,, 3) x
X
E.9 Winner-take-all STDP
While Contrastive Divergence uses MCMC sampling to approximate the GEM update, Nessler et al. [48] use particular generative model for which the posterior can be analytically calculated and resembles a simple winner-take-all neural circuit. Then, the authors derive their STDP parameter update approximation to the GEM algorithm. Because of this, one might imagine that the derived STDP update may, like the GEM algorithm, be -loca.. We will see below that this the case.
First, we define the generative model Pm used in the paper:
(r W)
1
F(ra.W)
(E.29)
Z
E(r,s, W)
Σ W
ΣV
r W 1588
(E.30)
where Z is the normalizing constant and the and s vectors contain binary random variables. Furthermore, in the network only one neuron r is assumed to fire at any given time (r; 5f 0 < r 0 Vk i). The inference distribution, conditioned stimulus can be caleulated as follows:
0z=0 EN W + EN Eye W
exp
Pm (rls, W)
(E.31)
Ei exp WWo + Eny Wiys/)
26

---

This probability distribution can be interpreted as kind of winner-take-all computation, dominated by the neuron with the greatest input current [48]. Samples from this distribution are used compute the weight updates:
A AW; ij ri (esse" Wis -)
(B.32)
A x r,e Wi -,
(E.33)
where e a positive constant. We now prove the following theorem assessing the p-locality of this distribution: Theorem E.9. The STDP update given by (STDP(p(X(O)) is -local
Proof. To see this, we first note that the gradient of the energy function E(r, W) with respect to the parameters is Pm-local by Property 2.6. Therefore, any variables contained within this will also be permissible under pm -locality.
For Wij. we have:
(E s, W)
(E.34)
OW
so that we know r; and are permissible for A AW; under Pmclocal;ty; further, the value of a
parameter itself, W sy. is always allowed under Pm -locallt. These the only variables which
A W.y kj depends, so this update is Ploca.
For W to, have:
(E s, W)
-
(E.35)
OW 40
so that we know r; is permissible for A AWa, oo By the same reasoning, this update also Plcal. may conclude that the full algorithm is Pm -lcal.
Since all updates are therefore Pmcll However, this proof does not have the same level of generality as for the previous algorithms, because the algorithm is only defined for single winner-take-aa network model.
E.10 Backpropagation
Theorem E.10. () =
p())) and p( ((X is defined by Eq. 3 (with X i3 with loss C(r), given by (er(pp())),, dr," Similarly, the updates for feedback alignment, weight mirror, and Burstprop are o! p-local, where e!") is given by their respective gradient approximations.
= r and θ = W), c!) p-local, where d!ll
the BP update for w!!) aL
Proof. As a first step, for clarity purposes we will demonstrate that backpropagation [28], is not p-local with respect to the simple feedforward neural network architecture we outlined above; we will subsequently demonstrate that it and its approximations do satisfy a particular notion of Sp-locality. For scalar loss function C(r²) and single parameter ( the backpropagation gradient is given by the negative gradient of the loss with respect to the parameter of choice, using the reparameterization trick [25, 24] to take for a single sample of (r(() Jm the mean (noiseless) mapping from (ll-1) >> r()) to be p(U) h (wW()) so that by Eq. 5, we have r(!) f(1) (") By the chain rule, gradient descent gives:
!W! x dl(r(() dF(L)
L dF(k) (x(b-1)) dF(k-1)
drl)) (w(!))
(E.36)
d
Based on analysis in Section 2.4, this update function is clearly not p-local, because the update depends on firing rates (4) for l: > I. However, while backpropagation is not in general p-local, any algorithm can be Sp-local: for example, we take Z, then by Definition 2.3, any parameter update can contain any variable in the graphical model p(Z). Taking S = Z is inherently vacuous: Sp-locality is only conceptually useful if we can cleanly reduce the number of variables included in S for a broad set of biologically relevant neural architectures. Fortunatcly, for backpropagation
27

---

operating on a feedforward neural network governed by Equation 5, we do not need to include every variable the network. By Equation E.36, we see for our example feedforward neural network that:
dc dr
Aw &
(E.37)
d
aw!
where de dF
de(((l)) dr
(Tl
(n(k) my de(A-1)
drle 1 (r
the derivative of the global loss func-
dF
dr
tion with respect to the individual mean neuron activation r Interestingly, aw the derivative
of the mean parameter for neuron r,-is a function only the parents of r(),, which are therefore the coparents of w ij To verify that this particular component of the weight update is p-local, we can compare its dependencies the score function, which is in this case p-local by Property 2.7. As noted in Section 2.4, the score function is given by:
r (
N(V!") o² 1 N' (V!") 1-1
e Olog W) ow
(E.38)
where v ! (1-1), Because the score function is p-local, any variables that it depends on are permissible for p-local updates. The score function depends on r. (!) and r(--1), whereas dr(")
dw!!)
depends only on r(t-1), It follows that this function is also p-local.
As we already discussed, de is not p-local because it depends neurons downstream of r
dr
However, if define an auxiliary random variable e, dC we see that because multiplies drl
e with a p-local function, w!!)
1j e p-local
Importantly, this does not mean that backpropagation is biologically plausible: this notion of locality provides clues to how e!") could be calculated approximated in the brain, and explicit calculation of gradients could not be possible due to the weight transport problem [20]. There are
(1) could be approximated by approximate credit many recent models that account for how e;
assignment signal e(") involving either random feedback synapses that project errors backwards through the network (feedback alignment [29]) or feedback synapses that dynamically adjust through local synaptic mechanisms so that e provides an unbiased approximation (e.g. weight mirror or Kolen-Pollack alignment [30], and BurstProp [31]). Each of these algorithms decomposes into nonlocal feedback term e(") and p-local term exactly the same way, and are consequently pocal.
E.11 Real Time Recurrent Learning (RTRL)
Consider an autonomous recurrent neural network whose directed acyelic graphical model is provided by the following equations (we will ignore stimulus-dependence for notational simplicity):
p(r[W) = p(r(0)) IIII ((.(())((t 1), W) -i
(E.39)
((r((t))rs(t 1), w) (W 1)),
(E.40)
where p(r( corresponds to some initial distribution of activity states. This probability distribution of firing rates corresponds to the following neural sampling dynamics:
r(t) h( (Wr(t - 1)) on)
(E.41)
where 17 ~ NN00 1). For this model, we have the following theorem:
Theorem E.11. Ifp(O) ( = =Ilg (xx) and p(X(@) is defined by Eq. E.39 (with X = r and θ = W). the RTRL update for W sj with a loss L(r(T)), given by Anrna (p(r is cJp-llccl, where
aL(r(T) and J = (J(t) aF(t.r) लह(
Br(t-1.r))
28

---

Proof. The directed graphical model corresponding to these dynamics is depicted in Figure 1b: as with backpropagation, we will use the score function for our graphical model to identify permissible variables. For single synapse, the score function is given by:
b pl W) OW ij
i log (r,,(((rrrrrt (t) - 1), W)
(E.42)
oW ij
(r:(t) h(V.(t)) H' ((Vi(t)rr(tt-))) 1),
(E.43)
o²
where V,(t) = War(t- 1). Thus p-local parameter updates for Wa may include W.. (t) and [rx(t- - W 0} vt. We will now compare these allowed variables to the RTRL update. As with backpropagation, we take F(t,r) (Wr(t - 1)). so that r(t) F(t, r) + on. The RTRL update minimizes a loss ((r(T)) via the chain rule [49, 21]:
AW x @C(r(T)) (T( OF(T) OW
(E.44)
or(t.r)
OF(t,r)
+ gr((t 1))
(E.45)
OW Ōr(t 1,r)
aW
g(r(t 1))a (N((v(())((t-)))
ifi
(E.46)
otherwise.
this second equation provides a recursive update equation which can stored online as a trial progresses. The g(r(t- - term is p-local, because it appears in Eq. E.43. However, JF([-1.) an N N Jacobian matrix, is not p-local, since it depends on all neurons in the network r(t 1) as well as all parameters Weneurons that do not directly synapse onto neuron r; and weights Wa Ar for k + are excluded from p-local updates by Property 2.1 according the DAG defined by Eq. E.39. Furthermore, as we have seen with backpropagation, in general the credit assignment signal ac(r(T)) "i(7")
0F((.r)
not p-local. Therefore, to characterize the Sp-locality of RTRL, we will have to proceed similarly to backpropagation, and define auxiliary variables to include in the set S.
As with backpropagation, define the auxiliary random variable found the Jacobians also violate p-locality, we will also define the set of auxiliary variables @r(t.)
ac(r(T))
dr(T) Because have
@F(t.r) is Jp-local Vt, and J = (t) de(r-1,r) With these auxiliary variables, we can see that 8W,; consequently, the RTRL update is eJp-local.
This is, course, not biologically plausible in any way. The set allows the parameters to have access to the state of the entire network, all time points, even from neurons that do not have any direct connections to the neuron whose synapse is being updated. Further, the entire error vector is required to compute the update. This is even less plausible than backpropagation, which only required access to e, However, the RTRL update is an important baseline for analyzing the locality properties of other learning algorithms that constructed approximations of it, namely e-prop and RFLO.
E.12 e-prop
Theorem E.12. !f =
((kx) and p(X(O) is defined by Eq. E.39 (with X = r and θ W), the e-prop update for Wij with loss L(r(T)), given by (cp [(r)) c;p-local, aL(r(T) where e; (T.
Proof. For e-prop [37], we will also consider networks constructed according to Eq. E.39. The update almost identical to the RTRL update, but several terms will be discarded, allowing the update to be e,p-local, opposed to the eJp-local update given by RTRL. The update is follows:
29

---

(C(r(T)) (T,)
AW bf (T) ow
(E.47)
(t, = N(V,())) (t,(t-1,r) (((()()))((--)) aw
(B.48)
OW ij
This update combines the neuron-specific credit assignment signal e, with local 'eligibility trace' (F.(t.E which performs approximate credit assignment by filtering and summing coactivity between 8W,5
neuron i and neuron j across timesteps. It is worth noting that the particular functional form of this eligibility trace is determined by our simplified RNN dynamics (Eq. E.39), which causes coactivity from previous timesteps to decay exponentially in proportion to the magnitude of the autapse W alternative neural network dynamics using continuous-tiin thresholds may alter the functional form of the eligibility trace [37], but do not fundamentally alter the p-locality properties of the update, Now, we only need show that the eligibility trace p-local.
dynamics, or adaptive neural firing
As with RTRL, we can observe that h'(V (t)) and ry 1) both appear in the score function for our RNN (Eq. E.43) for all timesteps, does Wa W.:. Because the score function is p-local, we know that these variables are all allowed under p-locality. The eligibility trace only depends on these terms, from both the current time step and, recursively, from previous timesteps. Therefore, the eligibility trace is p-local. The e-prop update is multiplication between e; and the eligibility trace, so by Def. 2.3 the update is e,p-local.
E.13 Random feedback local online learning (RFLO)
The RFLO update [36] is nearly identical to the e-prop update, except we replace e, with an approximate credit assignment signal é, (which replaces symmetric feedback weights with random connections, similar Feedback Alignment).
The update given by:
(T,
AW iy I é;
(E.49)
OW,;
or (t, r) ((V 0t - I,T) + (tV.
(E.50)
OW 1j
OW
Following exactly the same reasoning as with e-prop, we may show that this update is ê,p-local.
Theorem E.13. Ifp(@) = II p)) and p(X|O) is defined by Eq. E.39 (with X and O = W). the RFLO update for Wij with loss C(r(T)), given by (RFLo(p(T]W), C(x)) ē,p-local, where (L(r(T) (T,
E.14 Feedback-hased Online Local Learning Of Weights (FOLLOW)
The FOLLOW algorithm [50] is defined in terms of particular continuous-time LIF circuit with postsynaptic potential kernels. For simplicity, we will focus our analysis on a linear version of the same circuit, disregarding the dynamic postsynaptic potentials and input stimuli. Disregarding these features does not affect the p-locality properties of the FOLLOW algorithm, but it would certainly degrade its performance on tasks.
The network dynamics are given by:
r(t + At) = (n(((()))))))))))) e(t) +on
(E.51)
= - At r(()) + (Wr(t) kw)) At oon.
(E.52)
where f is an N x No random feedback weight matrix, k is positive constant, and e(() is an di-dinensional error feedback vector delivered at every timestep, with No the number of output dimensions. Because are in continuous time, we will assume that ~ (O, At).
30

---

Similar to RTRL, we can write the probability distribution for the network as:
p(rle, W) p(r(()) IIII p(r,(t (t))r(t),e(t),, w)
(E.53)
prr + (()) w) (()(r((),,(t)))) of
(E.54)
where p()( is some initial distribution of firing rates. Further, we can assume that the distribution of errors at timestep + A/ has any arbitrary distribution p(e(t + r()).
The update for weight Wi is given by:
A Wwy(1) ((t")(()) (t)..
(E.55)
Therefore, only the postsynaptic error current and presynaptic input are necessary to update the weights for given synapse in this type of network. Below, we will show that this update is p-local.
Theorem E.14. (()) Ih (Ok) and p p(X]@) defined by Eq. E.53 (with X (r, e) and
θ W), the FOLLOW update for Wij. given by Arw (p(r, el W)) is p-local.
Proof. To see that this is true, we need only show that the variables included in AW; ij are subsets of the variables included in the score function Blog plrpp/r These variables are permissible for p-local aW updates by Property 2.7. The score function is given by:
log p(r, e W)
plogp(r(( + (t)r W)
i logp(c(( + At)(r(t))
(E.56)
OW 4)
@W
OW i
Plogp(r,( + (t)) W)
Σ
(B.57)
OW ij
ri(l + At) - r:( + At
(-ra(1) + War(() + kW: e()
Σ
(t).
Ato²
(E.58)
Therefore, for weight AWaj ij. the permissible variables include: r.(() VI, any (4) such that W. + 0 (Vt), any (t) such that wifh fb # 0 (Vt), and the parameters W. and w The parameter update requires only (t) and w (t), which is a subset of these permissible variables. Therefore, the update is p-local.
F p-locality does not guarantee biological plausibility
It is very important to clarify the exact relationship between p-locality and biological plausibility. Except for some network-wide variables that theoretician may decide allow through particular choice of Sp-locality, we have generally shown that p-locality is overly permissive, in that a particular choice of p may allow parameter updates to include variables that an individual synapse may not have access to. Furthermore, p-locality does not restrict the network architecture defined by p to be biologically plausible. The best way to interpret p-locality is as follows: if p(X, +) defines a biologically plausible architecture and algorithm A is p(X, -locaa then the parameter update provided by A(p(X, O)) will be biologically plausible. There are many network architecture and parameter update combinations that may be biologically plausible without being proven p-local (e.g. explicit approximations to backpropagation [29, 30, 31]). as there are many combinations that are p-local without being biologically plausible. Below, we will show two important instances in which p-locality does not properly diagnose a combination of network architecture and parameter update as biologically implausible.
E1 Locality and architectural plausibility
The first example pervasive in neural network models of the brain: networks frequently violate Dale's 's law, which states that neurons in neural network are (for the most part [51]) either excitatory
31

---

(outgoing weights are positive) or inhibitory (outgoing weights are negative). but not both. In fact, in the simple network example we have provided (Section 2.4), neural firing rates are not constrained to be strictly positive, and outgoing synaptic weights are not sign-constrained. For this biologically implausible architecture, p-locality defines which variables are allowed to be included in individual parameter updates in way that is sensible (allowing only variables involving the postsynaptic firing rate and the firing rates of all pre-synaptic neurons), but says nothing about the aforementioned implausibilities of the network architecture. Similarly, the linear Boltzmann machine example provided in Appendix D does not constrain firing rates to be positive, and requires symmetric weights (We 6 W ji). which could not satisfy Dale's law while allowing connections between inhibitory neuron i and an excitatory neuron j (Wey > while W J < 0 Wij + W
These examples illustrate an important fact: p-locality focuses on the plausibility of updates given an architecture that has been predetermined to be acceptable. However, it is worth noting that if we were to impose these additional constraints for the proposed networks, the accepted variables determined by p-locality would not change.
F.2 Parameterizing probabilities with neural networks
Another important caveat when working with p-locality is that the random variables Z have to correspond to the relevant biophysical quantities of interest, e.g. neural firing rates X and synaptic weights θ If this is not the case, then p-locality can easily defy standard notions of biological plausibility. For instance, if we define probability distribution in terms of 3-layer neural network:
p X θ) (FF (F)()))
(F.1)
Then the score function of this distribution is given by:
Op(X θ
dloo Aloog(XXe X θ) (2) dr()) (r
(F22
d
dr
(F()) do do
This equation depends on X, which is the output of the network, even though + parameterizes F((). Therefore, if the random variables had been defined in as in Section 2.4, then this update would not be p-local. However, because it is the derivative of the score function, for any independent marginal p(O). it is p-local for this choice of random variables. Therefore, is important when working with algorithm such Wake-Slegg distribution p(X ()) that captures biologically plausible dependencies. When this is not done, as in [52, 53], the resulting updates have no correspondence to synaptic plasticity rules.
REINFORCE, that one chooses conditional probability
Note that this fact does not undermine the utility of p-locality as a concept. Our proofs for algorithms in Appendix E apply for any p(X as long as p(O) factorizes to 11. p())). Therefore, algorithms that have universal p-local properties will respect the variable dependencies implied by O( whether this distribution is plausible or not, which means that the algorithms will respect variable dependencies for all plausible network architectures.
32


Reviews:
[{"review_id": "iOfneQmsuu", "reviewer": "Reviewer_1DnT", "summary": "The authors propose \"Sp-locality\" as a formal definition of locality in models of synaptic plasticity, as well as two intermediate definitions, S-locality and p-locality. S-locality explicitly enumerates the set of variables which directly participate in a synaptic update for each synapse. p-locality replaces the explicit set S with the Fisher information of that variable (conditioned on all other variables to eliminate indirect dependencies) with respect to the synapse. Finally, Sp-locality expands this by measuring influence as defined by p-locality, additionally allowing a dependence of the synaptic update on an explicit set of variables as with S-locality (although eliminating the need for an exhaustive enumeration). Finally, the authors apply these definitions to a number of well-known models of synaptic plasticity, delineating four major classes of models and suggesting experimental predictions.", "questions": "- Is there other work attempting a conceptually similar grouping of plasticity rules based on the information used by synapses?\n- Can these definitions account for temporal locality or lack thereof? e.g. how would backpropagation through time or its biologically plausible approximations fit into this framework? \n- Is there a mapping between p-locality and S-locality? i.e. given a p-local rule (for a particular architecture), is there always a way of defining S-locality, and vice versa? (Is this even a useful mapping to make?)\n- Are there plasticity models that do not fit into this framework (excepting the trivial cases in which S=Z)?\n", "limitations": "The authors carefully analyzed the limitations of their definitions, although as far as I can tell these limitations severely handicap the practical utility of the proposed framework. Due to the basic science nature of this work, potential for direct societal impact is minimal. \n\n", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 2, "strengths": "I am unaware of similar work attempting to organize plasticity rules according to their locality, although I am interested to see what other taxonomies have been suggested. The paper is exceptionally well written, building up the concept of Sp-locality from simpler intuitive concepts, providing examples and intuitions, and expanding on these in the Appendices. The authors classify a wide range of models using their framework and show useful mappings between the graphical model and their definitions of locality. Overall, this paper is a useful initial effort to formally categorize \"local\" plasticity rules, and is a valuable conceptual contribution and starting point for follow-up work in this direction. \n\n", "weaknesses": "- Although the authors do a thorough job of applying their framework to a diversity of plasticity models, I am curious about further contextualization of the work within meta-topic of \"taxonomy of plasticity rules\"\n- As the authors point out, p-locality does not imply biological plausibility (nor vice versa). Sp-locality is an even weaker constraint, as it allows for not only p-locality but also dependence on an explicit subset S of network variables. These definitions punt the problem of \"biologically plausible plasticity\" to the problem of defining \"biologically plausible architecture,\" which is again often defined intuitively and ad-hoc. As such, the practical utility of this framework is unclear to me. \n- It is unclear to me how this framework adds power to experimental predictions beyond the ones made by the individual plasticity rules and their ad-hoc appeals to biological plausibility. I would be interested to see more discussion about this with explicit examples of predictions which can be made with this framework and followup experiments that can be performed (even conceptually, disregarding their practical feasibility), or even a retrospective \"post-diction\" of a known experimental phenomenon which could have been predicted by this framework. \n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your very comprehensive review; we will elaborate here on your criticisms and requests for clarification. All of our comments below will be incorporated into our subsequent draft. \n\nFor our response on previous related work, please see general comment [G1].\n\nIt is a genuine issue that our framework does not answer the question: “which network architectures are biologically plausible?”. Whether a mathematical framework could (or should) be constructed that would answer this question is unclear; arguably, that question can only be answered by careful anatomical and neurophysiological investigations. However, it's worth noting that Sp-locality does not completely punt the question of “biological plausibility” to that of “biologically plausible architecture”. A proof of Sp-locality necessarily comes with the class of models that the locality properties have been proven for (see Table 1). This helps us to make clear the generality & restrictions of various local learning algorithms and facilitate their comparison. In the ideal case, an algorithm would be Sp-local for any network architecture, because this would guarantee that an algorithm at least does not restrict a modeler to using networks that violate important biological plausibility assumptions. Of course, if an algorithm is only Sp-local for a given type of architecture then one is still left with the question, “is that type of architecture biologically plausible?”. But, critically, Sp-locality forces us to contend with this fact: that is its practical utility. Without it, researchers can (and often do) mask the biologically implausible architecture assumptions they are making when they claim that their algorithm is biologically plausible because it is “local”. Put another way, the utility of Sp-locality here is to eliminate any uncertainty as to the architectural assumptions being implicitly made in claims of locality. \n\nFor our response on experimental predictions made by our framework, please refer to general comment [G2].\n\nThis framework certainly can (and does) account for algorithms that do temporal credit assignment. For example, models that support general DAG architectures (in particular REINFORCE and Wake-Sleep) can also be used for learning through time, and these temporal network architectures do not disrupt our proof in any way because they are subsets of general DAG models (see Fig. 1b). We will add a discussion of these matters and highlight some additional specific algorithms, including  BPTT and several of its approximations (RFLO, e-prop), plus FOLLOW (Gilra & Gerstner 2017)--the derivations for these algorithms are very similar to the one provided for backpropagation (Section 3.3 and Appendix E.9). \n\nThere is always a mapping from a p-local rule to an S-local rule for a given network architecture–to see that this is true, we need only recognize that for every p-local rule, there is a finite set of allowed variables for each parameter. We may take $S_k$ for parameter $k$ to be equal to this set, thus demonstrating that the rule is also S-local. Of course, S-locality is in general radically less concise than p-locality, because without the machinery p-locality provides, it is not possible to automatically and cleanly generate a set of allowed variables, which would otherwise have to be done manually. It is also radically less useful because the set S is architecture-specific, and cannot be used for proving general properties of learning algorithms as we do in Table 1. By contrast, there is not always a mapping from an S-local rule to a p-local rule. This is because p-locality itself makes a hard commitment to an intuitive notion of locality defined on probability distributions, and does not accommodate additional variables that violate this intuitive notion.\n\nTo our knowledge, there are no normative plasticity models that do not fit into our framework. In fact, we have been unable to find normative plasticity models that do not fit into the four broad categories outlined in Table 1. However, it is worth noting that p-locality becomes less useful for non-normative (e.g. phenomenological or mechanistic) plasticity rules, where models are normally highly architecture-specific, and there is less value in constructing an architecture-general notion of locality, because in these cases there is no proposed generic optimization algorithm (e.g. REINFORCE) producing different plasticity rules for different neural network architectures.\n"}, {"review_id": "0Le6XOvxGI", "reviewer": "Reviewer_X8HR", "summary": "The paper introduces a general definition of locality of updates in neural networks, which is an important requirement of biological plausibility. Variants of locality requirements allow to describe classes of networks and algorithms. Several important algorithms are analyzed through this lens. ", "questions": "The authors claim there is lack of clarity about locality for some models. Can you give an example?\n\nDefinition 2.1: line 87, what are you trying to say by underscoring the word 'direct'? what would be indirect?\n\nFig. 1b Node r_j for t_2 is not a child or coparent of W_{ij} but not excluded.\n\nColoring in figures can be guessed, but better be explained in words.\n\nEq.4 How would the definition 2.2 work when $h$ is not differentiable, e.g. ReLU?\n\nline 239 A(p(X, \\Theta)) = f(Z) - it looks like update function is somehow fully determined by the probability distribution? Please define the meaning of A(p(...)) here and in theorems 3.1, 3.2.\n", "limitations": "No problem nere", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Definition of locality (and variants) is novel (to my knowledge), considerations are mathematically sound.\n\nThe paper outlines several patterns of locality in existing algorithms, which deserves interest.", "weaknesses": "The need for the general definition of locality is not substantiated sufficiently. Was that need already expressed in the literature? Were there other attempts? Is there an example where such definition would resolve a controversy?\nWhenever a model is formally defined, its biological plausibility (which of course goes beyond locality) is up for debate. However within the model the information available for parameter updates must be clear from the definition (or I can't think how it can not be). Therefore I am not convinced that suggested framework will help \"guide claims of biological plausibility\", as the paper says.\n\nIt is not quite convincing that introducing probabilities is worth the trouble. After all, the answer to locality question is always binary: yes or no. It seems that definition through graphs (directed or undirected) would be sufficient in overwhelming majority of cases, and whenever the probabilities are already defined, the graphical model can be referred to.  \n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "First, we thank the reviewer for their detailed feedback on our manuscript. We will make sure to add relevant material in relation to our responses below in our subsequent draft.\n\nFor a discussion of the current state of the field, and an explanation of what our framework adds to discussion surrounding experimental validation of existing models, please refer to our general reviewer responses [G1] and [G2]. But, to expand on these points a little bit here, there are two key reasons that these formal mechanisms were needed. First, in contradiction to the reviewer’s intuition, it is not the case that the information required for model updates is always apparent based simply on the definition of the learning rule. Consider, for example, the predictive coding algorithm. The update rules themselves appear to be purely local, and at first blush, they provide the impression of clear biological plausibility. Yet, when analyzed under the lens of Sp-locality, we see that these apparently local equations depend on an architectural assumption of maximum a posteriori gradient descent dynamics, which is biologically questionable. Thus, one benefit of Sp-locality is precisely that it forces us to actually contend with the assumptions being made to achieve locality. Without this formal tool it can be easy for researchers to convince themselves that something is “biologically plausible” because it is local, while ignoring the assumptions required to achieve that locality. Second, our framework based on probabilities is useful beyond a basic verbal description because it provides a formalized and standardized method for describing the biological plausibility of a given plasticity model–beyond this, it also abstracts away details that are not useful for comparison across different network architectures and algorithms (see [G2]).\n\nFor a full discussion of our particular choice to use probability distributions to ground our definition of locality, please see [G3]. Here we will specifically justify our choice to use stochastic—rather than deterministic—computation graphs. More than half of the algorithms listed in Table 1 were originally derived in terms of probabilistic networks. Given that the probabilistic formulation is more general (we can always look at no-noise asymptotes for the deterministic case), it makes sense to work in probabilistic terms. Further, the probability distribution provides the cleanest mechanism for deciding how to structure the edges and nodes of the computation graph, as we describe in [G3].\n\nFig. 1b Node $r_j$ for $t_2$ is not a child or coparent of $W_{ij}$ but not excluded. This is a good catch, thank you! It is a coparent for $W_{ij}$ at the next time step--we'll add a line that is implicitly connecting to the next time step to clarify this. We will also explain the color coding in more detail.\n\nWe only need our update f in Eq. 2 to be differentiable at a single point and for that derivative to be nonzero, so for a ReLU this would be fine (it's differentiable and nonzero for all x > 0). In this case our assumption of differentiability is not so important. The Heaviside function is a better example, because at the only point at which it does depend on a variable, the derivative is not well-defined. In this case, there is no clear way for the derivative operation to detect a functional dependence. A way to relax our assumption of differentiability would be to require that there are two values of $Z_i$ such that $f(Z_{i1}) \\neq f(Z_{i2})$ on the left side. This would not disrupt anything for S-locality, but would potentially disrupt our proofs for the properties of p-locality, so we chose to require differentiability throughout. Another alternative would be to use a function that very closely approximates the heaviside function but preserves its differentiability (e.g. sigmoid(k x), where k >> 1).\n\nClarifying the meaning of $\\mathcal A(p(X, \\Theta)) = f(Z)$. The notation here indicates that an algorithm $\\mathcal A$ receives a probabilistic network architecture $p(X, \\Theta)$, and outputs a parameter update function $f(Z)$. This update function changes if the probability distribution changes. For instance, the REINFORCE update Eq. 6 changes if we change our choice of nonlinearity $h$ which is used to parameterize our probability distribution over network states. One of our highly nontrivial results is that we can show the REINFORCE update will be Rp-local no matter what choice of probability distribution p we make."}, {"review_id": "OPN9CNCBTr", "reviewer": "Reviewer_p6bX", "summary": "The paper describes a systematic and mathematically formal method for categorizing learning algorithms into different types and degres of locality. It also applies the method to certain such algorithms and determines their type of locality.", "questions": "\"We subsequently use this framework to distill testable predictions from various classes of biologically plausible synaptic plasticity models that are robust to arbitrary choices about neural network architecture\"\nThis is a quote from the abstract. Could the authors please elaborate on this? What exactly are the testable predictions? Perhaps this refers to the last paragraph of the paper, but that paragraph is not entirely clear to me, and it also seems like it does not make any specific prediction.\n\nIf the authors could address these issues both in the revised manuscript and in their rebuttal here, I am open to re-evaluating the submission.", "limitations": "There is no discussion of the paper's limitations in the paper.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The authors chose to work on a topic that is indeed very important, and in a field has been very much in need of systematic evaluation. The claims of biological plausibility and locality in the recent literature have been increasing and widening, so a method to validate such claims has been necessary.\n\nThe fact that the paper does not merely provide a verbal definition of types of plasticity but rather it attempts to make it mathematical is very interesting.\n\nThe framework is mostly reasonable, judging by its results when applied to the categorization of specific learning rules.", "weaknesses": "The weakest aspect of the framework presented here is that it falls victim to one of the issues that it intends to resolve.\n\nMore specifically, algorithms such as equilibrium propagation or predictive coding require that the network reach a global equilibrium through multiple forward and backwkard propagations before a training example completes its updating action on the network parameters. This is arguably less local than even standard backpropagation, as it requires not only that global signals propagate through the network once, but indeed multiple times. This has been recognized in recent works (https://iopscience.iop.org/article/10.1088/2634-4386/aca710, https://openreview.net/forum?id=8gd4M-_Rj1) and this recognition has even led to a formal proof that the time-complexity of these algorithms is lower-bounded by that of backpropagation (https://arxiv.org/abs/2304.02658). Therefore, this type of locality is clearly not the same as that of, for example, Contrastive Divergence for Boltzmann machines or Deep Belief Networks which do not presuppose such a global and multilayer equilibrium. In turn, even in the cases where such an equilibrium requires iteration only within individual layers and where greedy layer-wise training is possible, e.g. in Boltzmann machines and contrastive Hebbian rules, the final weight update cannot be said to be equaly local to that of simpler Hebbian rules that do not require such iterative processes or feedback, and complete the update truly locally in a single timestep. The distinction of these categories of locality is crucial. Disregarding these in a paper that claims to resolve such confusions risks reinforcing and perpetuating them instead.\n\nThe above issue of the paper might have been caused by the fact that many of the simplest and most local learning rules are not normative but rather are based on heuristics, whereas the presented framework is concerned with normative rules. However, there do exist such rules that are normative, eg the type of STDP in Nessler et al., NIPS 2009/Plos Comp. Biol. 2013; the work of Pehlevan & Chklovskii, NIPS 2015; and SoftHebb in Moraitis et al., Neuromorph. Comput. Eng. 2022. In fact, what appears to be the current state of the art in biological plausibility of deep learning and in performance of bio-plausible deep learning (Journé et al., ICLR 2023) is one of the above normative rules, namely SoftHebb.\n\nTherefore, I recommend that the authors adjust their framework to account for the substantial differences in locality between the three categories of algorithms that are exemplified by (a) Equilibrium Propagation, (b) Contrastive Divergence/Boltzmann Machine, and (c) SoftHebb or STDP. This should be accompanied by making sure that examples of all these categories are included in Table 1.\n\nAs a minor note, the paper's Table 1 mentions Boltzmann Machines as an algorithm, but BMs are models. I believe Contrastive Divergence is meant by the authors, and if so it should be corrected.\n\nMore generally, the fact that the current formalization suffers from similar issues as other less formal claims in the literature, reveals that perhaps the mathematics in this case are superfluous, since its conclusions depend mainly on the assumptions, which in turn depend on the categories that the authors intend to achieve. In other words, I suspect that the main benefit of a systematic categorization could be achieved without mathematical terms. Could the authors please comment on this? What do they regard as added value from the use of mathematics instead of mere natural language and logical arguments in this case?\n\nRelated to this, the paper would be strengthened significantly if both the framework and the resulting categories of locality could be described in a dedicated paragraph in natural language, with an intuitive explanation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their very detailed comments! We will respond to your specific comments here, and for questions asked by multiple reviewers, we will refer you to the appropriate response in the General Comments section [GXX]. All of our comments below will be incorporated into our subsequent draft.\n\nWe agree with the reviewer that a good framework for analyzing locality should contend with the difference between algorithms that require MCMC sampling and those that do not. If, as the reviewer suggests, our framework masked this distinction that would indeed be problematic. However, that is not the case. Indeed, our definition of Sp-locality requires one to determine the architecture required for locality, and some architectures will not require MCMC (such as a DAG), while others will (such as an energy-based model). Thus, Sp-locality does not ignore the question of whether MCMC sampling is required or not. We will be sure to clarify this in our revisions.\n\nHowever, a related though slightly different question that the reviewer raises is how many samples of MCMC are required by an algorithm for effective training. Here, we agree with the reviewer that our framework does not fully account for this. But, it does partially. Indeed, we should clarify the similarities and differences between Contrastive Divergence, Equilibrium Propagation, and more typical Hebbian rules in this respect. In particular, we note that the Nessler et al. NIPS 2009/Plos Comp. Bio 2013 papers construct their learning algorithm as a very close approximation of the generalized Expectation Maximization (GEM) algorithm, which is as a consequence $p_m$-local (see Table 1), where $p_m$ is defined by the generative model used in these papers. While generative models typically have neither analytically tractable nor biologically plausible inference distributions, by making a careful choice of generative model, these authors are able to perform exact inference through a type of winner-take-all neural circuit. As a consequence, they do not need to rely on slow MCMC dynamics or MAP inference, and are also able to avoid offline ‘sleep’ phase sampling (this is why this model does not require information about a global clamping variable $\\gamma$). This sets this family of algorithms apart from both Contrastive Divergence and Equilibrium Propagation, and our framework captures this. Similarly, SoftHebb derives its locality via very similar mechanisms without requiring sampling from multinomial variables, and again, this can be identified via our formalisms. We will add these models to Table 1, and will note their relationship to the GEM algorithm in the appendices. \n\nBut, in line with the reviewer’s point, it is true that Contrastive Divergence does not require full mixing for MCMC sampling, unlike Equilibrium Propagation, and yet this fact does not affect the p-locality of the Contrastive Divergence learning rule versus Equilibrium Propagation or Predictive Coding (though it does affect the supported architectures for the learning rule, see column 2, Table 1). This raises an interesting possibility: would it be possible to extend our framework to not only distinguish models that require MCMC sampling from those that don’t, but also to distinguish models that require only a little bit of MCMC sampling from those that require a lot? We think this is a fascinating potential direction for future work building on our framework here. We will add a note on limitations highlighting this point to our paper, and emphasize that future work should address this distinction. To ensure that we are not adding to any confusion in the literature we will also add a note in column 2 Table 1 to emphasize that Contrastive Divergence is somewhat more flexible than predictive coding and equilibrium propagation in terms of its time complexity, because it requires only K-Step MCMC sampling where K is a small integer.\n\nFurther, thank you for catching our misuse of terminology. We will change our use of ‘Boltzmann Machine’ to ‘Contrastive Divergence’ in Table 1 and in the appendices.\n\nFor a discussion of the relationship between our approach and a more manual/linguistic approach, please refer to response [G4].\n\nWe provide a detailed explanation of the different Sp-locality categories, as well as their experimental predictions in General Comments [G2]. For a broad description of the approach in general, we refer to Lines 45-59 of the main text. We will use these to expand and clarify the final paragraph in the Discussion section.\n"}, {"review_id": "vyhyendcuY", "reviewer": "Reviewer_GWia", "summary": "Many biologically plausible learning algorithms have been developed in the recent years but much less effort have been deployed to compare them and understand how they could yield experimentally testable hypothesis. The present work introduces a framework that allows comparing existing algorithms. In particular, the authors introduce the concept of Sp-locality which highlights the quantities needed to be available to synaptic plasticity, the p part focusing on quantities made available by neural computations and the S one on additional signals learning signals. Different groups of plasticity rules arise from the proposed analysis, helping formulating empirically falsifiable hypotheses.\n\n\n**Score increased from 6 to 7 after the rebuttal**.", "questions": "**Note:** all the following questions are based on my current (incomplete) understanding of the paper.\n\nIn a feedforward neural network, a plasticity rule for $W^l_{ij}$ (synaptic weights between layer $l-1$ and $l$) that depend on $r^{l-1}$ (activity of the **entire** $l-1$ layer) and of $r^{l}_j$ is consider $p$-local. Is this compatible with any notion of biological plausibility? This point makes me wonder whether the notion of $p$-locality is fully aligned with locality in physical space.\n\nWhat are the main differences of the proposed approach with an alternative one focusing on the computational graph of neural computation? Such an approach would, for example, not suffer from the FF issue described above. It would start from the neural computations performed and end up with the learning rules. Of course, one downside is that it now depends on the details of neural computation.\n\nHow would the Sp-locality framework extend to algorithms attacking the temporal credit assignment problem, e.g. e-prop of Bellec et al. 2020 or FOLLOW of Gilra and Gerstner 2017?\n\nFinally, which conclusions the framework enables establishing that a more vanilla approach that just analyze the learning rules “manually” does not achieve?", "limitations": "Yes", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "The paper is technically sound, introducing formal definitions of locality and thoroughly examining their implications. It is well written, with examples and illustrative figures facilitating reader comprehension of the newly introduced concepts. The analysis conducted allows a straightforward comparison of various learning rules. Furthermore, the authors transparently outline the framework's limitations.\n\nThose points make the paper valuable to the NeurIPS community, particularly due to the lack of prior work on the question it addresses.", "weaknesses": "Some of the properties deriving from Sp-locality do not seem particularly aligned with what one could picture of biological plausibility. As this may result from a misunderstanding from my side, I address those points in the question section. I am happy to improve my score once those questions are addressed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your feedback, here we will do our best to clarify the points that you raised. In some cases we will refer to the General Comments [GXX] if your question was also raised by other reviewers.\n\nRegarding your question about whether or not p-locality is fully aligned with our intuitions about locality in space: in our example from Section 2.4, we do see that updates in a MLP network are dependent on all neurons in the preceding layer. However, this is because MLP networks are all-to-all connected. If a neuron in layer $l-1$ were not directly synapsing onto & communicating with neuron $r_i$, then no synaptic updates would be able to depend on it under p-locality. We do see that some update rules depend on ALL presynaptic neurons however, rather than only the particular synapse’s presynaptic input, $r_j$. This is acceptable, however, because in this example all neuronal inputs are being summed to contribute to the membrane potential $V_i$. Within neuroscience it is well-known that synapses onto the same neuron can interact and share information (i.e. heterosynaptic plasticity, a well documented phenomenon), and if the update depends on the sum of all activity in particular, then it actually coincides quite well with our basic intuitions about spatial locality in neurons (since the sum of activity will be reflected in back-propagating action potentials in the dendrites). \n\nHowever, as we do note in considerable detail in Appendix F, when unrealistic or over-simplified networks are used, p-locality does not necessarily correspond to our physical, biological notions of locality. One benefit of p-locality is that the more realistic you make your biophysical neuron model, the more realistic the p-locality constraints become (which is why the examples in Section 2.4 and Appendix D produce sets of allowed variables under p-locality that closely correspond with physical, biological intuitions).\n\nFor a discussion of the relationship between our formulation and one focusing on the computational graph, please refer to response [G3].\n\nThis framework certainly can (and does) account for algorithms that do temporal credit assignment. For example, models that support general DAG architectures (in particular REINFORCE and Wake-Sleep) can also be used for learning through time, and these temporal network architectures do not disrupt our proof in any way because they are subsets of general DAG models (see Fig. 1b). We will add a discussion of these matters and highlight some additional specific algorithms, including  BPTT and several of its approximations (RFLO, e-prop), plus FOLLOW (Gilra & Gerstner 2017)--the derivations for these algorithms are very similar to the one provided for backpropagation (Section 3.3 and Appendix E.9). \n\nFor a discussion of the relationship between our approach and a more ‘vanilla,’ manual approach, please refer to response [G4]."}]

Author rebuttal:
#### General comment

We would like to thank the reviewers for their extremely helpful feedback. Here, in the global reply, we will address critiques that were raised by multiple reviewers. Importantly, we are grateful that most reviewers recognized the importance of a formal framework to study learning algorithm locality. We believe our work is an important first step in that direction, and that it has the potential to be used widely. We also note that our manuscript is considerably improved thanks to excellent reviewer feedback. As such, we believe it is timely, and that it is worthy of sharing with the NeurIPS community.

[G1] Previous work. 
First, in our introduction we will cite other papers that have attempted to construct taxonomies of locality in the past and clarify our unique contributions. To our knowledge, no previous study has attempted to construct a mathematical framework for locality, but some papers have reviewed locality as a whole and discussed potential ways to test differences between different proposed normative plasticity rules. In particular, the four p-locality categories that we uncover with our analysis loosely correspond to those identified in Lillicrap et al. 2020. But, our mathematical framework provides additional useful distinctions, e.g. it identifies target-based, generative learning algorithms like Wake-Sleep as distinct from error-based learning algorithms derived from backpropagation, because these algorithms derive their locality from very different principles and make different assumptions about the nature of feedback. In addition, Marschall et. al 2020 complements our approach by discussing constraints on the allowed computations, memory, and temporal complexity of bio-plausible temporal credit assignment algorithms and taxonomizes existing approaches accordingly. These amount to constraints on the network architecture itself, and constraints on the types of operations allowed in computing updates (e.g. matrix-vector multiplication), whereas we focus on allowable variables for updates, agnostic to the update’s functional form.

[G2] Clarifying experimental predictions. 
We discuss this point briefly on Lines 351-55, but will elaborate here (and in our revised discussion).  Critically, Sp-locality abstracts away details that are not important for testing predictions and helps identify those that are. For example, suppose we are working with the feedforward network model of Section 2.4. Sp-locality tells us that the REINFORCE (Eq. 6) and Wake-Sleep algorithms (Eq. 8) are equivalent with respect to their ‘p’ in p-locality. This tells us that the set of allowed local variables under p-locality (e.g. pre- and post-synaptic information) are not the best targets for experimental testing between these algorithms, and instead the focus must be on the variables included in ‘S’.
Turning to this, we have identified four different kinds of Sp-locality, which vary based on the choice of set S (S = the reward signal $R$, the empty set, the global clamping variable $\gamma$, or the error feedback signal $e_i^{l}$). REINFORCE is of the first kind, and thus predicts that a global reward signal will modify synaptic plasticity. In contrast, Wake-Sleep is of the third kind, and thus predicts that there is a  global clamping variable $\gamma$ that switches the network between different modes of synaptic plasticity independently of reward. Therefore, across these different Sp-locality classes we have very different prescriptions for what to look for experimentally in terms of signals that regulate synaptic plasticity globally. 

[G3] Probabilistic versus Graphical locality. 
Several reviewers have suggested that we might alternatively formulate locality in terms of the graph of computations performed by a given network, rather than in terms of a probability distribution over network states. However, within each probabilistic node in our formulation there are several computations being performed (linear matrix + pointwise nonlinearity in the MLP case). A graphical approach would have to arbitrate which collection of computations counts as a ‘node’ on the graph. There are several different ways to do this: one way is equivalent to our approach, where the graph is taken to be a minimal DAG or UG which characterizes the distribution p. We demonstrate that this graph can be used to read out the allowed variables under p-locality (Properties 2.1 and 2.2), and we find that the reviewers are certainly correct that these properties are easier to use than the basic definition of p-locality itself. However, the definition of p-locality is still required to justify these properties and to unify them across different graph types (DAG vs. UG). Other approaches for defining the appropriate computation graph would be problematic, primarily because they would be more arbitrary, and consequently could not be used to prove general locality properties of normative learning algorithms. 

[G4] Language versus math. 
With any mathematical framework, as several reviewers have noted, it is critical to ask whether  natural language would have sufficed. In our case, the principal added value of Sp-locality is that it disentangles the locality properties of an algorithm from the specific network architecture used in the model, by exploiting mathematical  insights into how algorithms actually achieve locality (Properties 2.5-2.7). This allows us to focus on properties of normative plasticity algorithms as a whole, without nitpicking details about their particular instantiations for particular model choices, something that could not have been done without our mathematical formalisms.


Meta-review: Synaptic plasticity rules are essential in (computational) neuroscience to understand learning in nervous systems. Plasticity rules are often termed "biologically plausible", however, a clear concept for this classification is missing. One particularly important feature in this respect is locality of the rule in space and time. The authors propose a formal definition of different classes of locality. It is used to propose testable predictions of plasticity models.

All reviewers agree that the manuscript is very well written. The approach is novel and it is addressing a question that is highly relevant for computational neuroscience. Reviewers note that the paper is mathematically rigorous and that the framework is well-motivated. The framework is applied to a wide range of models which provides useful categorization and insights.

One concern was that a locality category may be too broad as an equivalence class, including rules of rather different locality. While a full resolution was not possible within the discussion period, the reviewer acknowledged the suggested improvements and voted for acceptance.

In summary, the manuscript proposes a very interesting framework to understand locality in plasticity rules in a more rigorous manner. Its main strength is its novelty.
Decision: Accept (poster)