Title: Formalizing locality for normative synaptic plasticity models
Venue: NeurIPS 2023
Accepted: yes
Keywords: synaptic plasticity

computational neuroscience
Abstract: In recent years, many researchers have proposed new models for synaptic plasticity in the brain based on principles of machine learning. The central motivation has been the development of learning algorithms that are able to learn difficult tasks while qualifying as "biologically plausible". However, the concept of a biologically plausible learning algorithm is only heuristically defined as an algorithm that is potentially implementable by biological neural networks. Further, claims that neural circuits could implement any given algorithm typically rest on an amorphous concept of "locality" (both in space and time). As a result, it is unclear what many proposed local learning algorithms actually predict biologically, and which of these are consequently good candidates for experimental investigation. Here, we address this lack of clarity by proposing formal and operational definitions of locality. Specifically, we define different classes of locality, each of which makes clear what quantities cannot be included in a learning rule if an algorithm is to qualify as local with respect to a given (biological) constraint. We subsequently use this framework to distill testable predictions from various classes of biologically plausible synaptic plasticity models that are robust to arbitrary choices about neural network architecture. Therefore, our framework can be used to guide claims of biological plausibility and to identify potential means of experimentally falsifying a proposed learning algorithm for the brain.

Formalizing locality for normative synaptic plasticity models

Abstract

In recent years, many researchers have proposed new models for synaptic plasticity in the brain based on principles of machine learning. The central motivation has been the development of learning algorithms that are able to learn difficult tasks while qualifying as 'biologically plausible'. However, the concept of a biologically plausible learning algorithm is only heuristically defined as an algorithm that is potentially implementable by biological neural networks. Further, claims that neural circuits could implement any given algorithm typically rest on an amorphous concept of 'locality' (both in space and time). As a result, it is unclear what many proposed local learning algorithms actually predict biologically, and which of these are consequently good candidates for experimental investigation. Here, we address this lack of clarity by proposing formal and operational definitions of locality. Specifically, we define different classes of locality, each of which makes clear what quantities cannot be included in a learning rule if an algorithm is to qualify as local with respect to a given (biological) constraint. We subsequently use this framework to distill testable predictions from various classes of biologically plausible synaptic plasticity models that are robust to arbitrary choices about neural network architecture. Therefore, our framework can be used to guide claims of biological plausibility and to identify potential means of experimentally falsifying a proposed learning algorithm for the brain.

1 Introduction

Over the last several decades, computational neuroscience researchers have proposed a variety of 'biologically plausible' models of synaptic plasticity that seek to provide normative accounts of a variety of learning processes in the brain-these models aim to explain how modifications of cellular properties such as excitability and synaptic strengths can improve performance on a variety of important tasks for an organism, using only the information 'locally' available to a real neuron in the brain. However, there is no consensus on what 'biologically plausible' really means. This is due to a lack of any precise definition of locality, with individual studies adopting different, architecturespecific heuristics that are often not explicitly articulated; further, at an experimental level, the precise quantities available to neurons for computing synaptic updates are still up for debate [1, 2], and likely vary across cell types [3, 4, 5] and brain regions [6, 7, 8, 9, 10].

Guillaume Lajoie †

As an example, some studies postulate that global neuromodulatory signals containing reward information could be available at individual synapses, in addition to the more traditional pre- and post-synaptic information typically required for Hebbian plasticity [11, 12, 1]. Within this broad class, individual studies differ in their assumptions about this reward information (e.g. is it completely global [13] or potentially distinct for different layers or regions [14]) and in the precise details about the pre- and post-synaptic signals available (e.g. is it spike-timing [15] that is available or spike-rate [16]?). Furthermore, there are many more signals that could theoretically be available at synapses: for example, other studies propose that individual pyramidal neurons could receive detailed error (or target) feedback information at their apical dendrites [17, 18, 19, 2] which can be used for learning. Previous work has taxonomized different normative plasticity models according to these feedback signals [20]. Within this space of normative plasticity models, it can be difficult to identify which features of a given model constitute necessary, testable predictions that can be used for experimental verification-i.e., if this information were not available at a synapse, learning with the given algorithm would be impossible-and which features are due to arbitrary choices that may vary with more or less realistic neural network architectures.

As a consequence of these heterogeneous conceptions of locality and given the advancements in the field, it is arguably necessary to develop a clear and precise way to identify what variables synapses should have access to in a learning algorithm, whenever claims of locality are made. In this paper, we focus on formalizing this process. Our central contributions are as follows:

We develop an architecture-independent formal framework for locality that we term S p -locality, which requires one to precisely specify the set of variables that synapses are assumed to have access to within a model: different choices of allowed variables for the set S produce different classes of locality. Importantly, this definition can be used for any learning algorithm applied to any stochastic neural network.

We use our framework to group existing plasticity models into different locality classes. Intriguingly, we find that many different algorithms derive their locality from similar principles, even though they belong to different locality classes.

We show that our different locality classes make different experimental predictions, which makes it possible to identify which algorithms can be cleanly distinguished experimentally and which cannot.

2 Formalizing notions of locality

Our ultimate goal is a formal definition of locality that makes it easy to identify which variables are locally available to synapses in a learning algorithm within any specific model architecture. However, this can be challenging. For example, when we consider a single-compartment neuron model it seems reasonable to assume that the postsynaptic voltage is locally available to synapses. In contrast, in a multi-compartment model, voltages can be different in each compartment, and synapses likely only have access to voltages within nearby compartments. Ideally, we would avoid a definition that simply requires us to list all of the variables that must be made available to the synapse, as that would carry little conceptual weight. However, we will begin by taking this approach, as it will help us to build the groundwork for a more conceptually useful definition in what follows.

̸

̸

Notation. We have X as a random vector in R N X denoting network variables (e.g. voltage, spikerate, inputs, etc.), and Θ as a random vector in R N Θ denoting network parameters (e.g. synaptic weights, spike threshold, etc.). We combine these as Z = [ X , Θ ] -the concatenation of the two vectors. We will use Z ∼ p ( Z ) to denote a random vector with support Ω ⊆ R N Θ + N X , but will also overload this notation to indicate a vector subject to either universal or existential quantification (e.g. ∃ Z s . t . d f ( Z ) d Z = 0 ) or a dummy variable in an expectation (e.g. E p ( Z ) [ Z ] = µ ). We will use the subscript ' = i ' to denote removal of the i th element of a vector; for example Z = i = [ Z 1 , . . . , Z i -1 , Z i +1 , . . . , Z N X + N Θ ] ⊤ , and will occasionally use the more general notation Z = ν , where ν is a set of indices, to denote the vector with the associated set of elements removed.

̸

2.1 S -locality: graph-specific locality based on a predefined set of available variables

We begin with a definition of locality that formalizes the concept of a set of variables, S k ⊆ Z , to which the k th synapse can have access in order to compute synaptic updates (e.g. some neural voltages,

reward signals, etc.). We then consider the set of these sets across all synapses, S = { S 1 , . . . , S N Θ } . This inspires the following definition:

Definition 2.1. Given parameters Θ ∈ R N Θ , random variables X ∼ p ( X | Θ ) ∈ R N X , and function f ( · ) : R N X + N Θ → R N Θ , the update function f ( Z ) is S -local with respect to some set S = { S k : S k ⊆ Z } if and only if ∀ k, i :

̸

In short, S -locality requires that if parameter update f k ( Z ) has a direct dependence on Z i , then Z i must be included in the set S k of 'allowed' variables for that parameter. For example, in the classic approach to locality in a neural network, for any synapse W ij ∈ Θ , we might set S ij = { r i , r j } ⊆ Z , allowing any S -local parameter update under this definition to use pre- ( r j ) and post-synaptic ( r i ) firing rate information, and nothing else. If we wanted to allow a third factor such as a global neuromodulatory signal, R , to project to diffusely to every synapse in the network, as in rewardmodulated synaptic plasticity rules, we might take S ij = { r i , r j , R } . In this manner, for any model architecture, we can manually construct a set S ij that defines our notion of locality for each synapse.

While our definition is intuitive and flexible, it is cumbersome: for every parameter we have to explicitly define which variables are local. This means that S -locality is architecture-specific, requiring an evaluation of which S k is an acceptable set of variables for each synapse's update rule. For example, if we switched from a single- to a multi-compartment model, we would now have to modify S ij to accept only the voltage of the compartment where the synapse is located. Ideally, our definition would make it obvious whether a learning rule is local in both models without having to redefine the list of which variables are local. To do that, we will use the structure of the distribution p ( X , Θ ) to construct a definition of locality that captures intuitions across network architectures.

2.2 p -locality

Despite its ability to concretely define locality for any model, S -locality is architecture-specific and cumbersome: in this section, we will develop an alternative operationalized notion of locality, called p -locality (where p stands for 'probability"). Our idea is to formulate locality in terms of adjacency in the computational graph underlying network dynamics. To do this we will leverage conditional dependencies in the joint distribution p ( X , Θ ) to generate an architecture-general concept of locality 3 . This is considerably more useful as both a conceptual tool and organizing principle for normative synaptic plasticity models. However, we will still call on S -locality later to formulate our final locality definition: S p -locality.

Further, it is important to note that for this definition, as with S -locality, we do not restrict the functional form of parameter updates f k ( · ) -we only restrict which variables they are allowed to depend on. This is in contrast to previous, complementary work which has focused on defining allowable local computations and memory complexity constraints for 'biologically plausible' learning algorithms [21]. Our decision not to restrict f k ( · ) enables us to abstract away the details of 'allowed' subcellular computations, and will consequently enable us to make much more general statements about the locality properties of normative plasticity algorithms as a whole, without having to focus on the specifics of particular models of neural dynamics.

̸

Definition 2.2. p-locality: Consider a probability distribution p ( Z ) over network activity variables and parameters, and an update function f : R N X + N Θ → R N Θ , mapping variables and parameters to parameter updates. Assume that ∂p ( Z i | Z = i ) ∂ Θ k and ∂f k ( Z ) ∂ Z i exist for all Z ∈ Ω and i, k indices. The update function f ( Z ) is local with respect to p ( p -local) if and only if ∀ k, i :

̸

̸

̸

To unpack this definition and render it more intuitive, as with S -locality, ∂f k ( Z ) ∂ Z i indicates a test to see whether the k th index of f has a direct dependence on a particular variable Z i -we assume

a

b

c

...

̸

that f k ( · ) is differentiable primarily for simplicity, and differentiable approximations can always be substituted in practice if f k ( · ) happens to not be differentiable. If there is a dependency, then we want that associated random variable Z i to be considered local to Θ k . To measure this, we have selected the Fisher Information of p ( Z i | Z = i ) with respect to Θ k . This measurement quantifies direct influence. For example, if Z i is a neuron downstream of neuron Z j , and Θ k is a synapse onto Z j , then by conditioning on Z j , Θ k is no longer able to have any effect on the conditional probability distribution of Z i through its influence on Z j . In other words, the statistical influence of Θ k on Z i is mediated through the influence of Z j on Z i , and thus it is indirect. By conditioning on Z j , we seal off one pathway of indirect influence. By conditioning on all Z = i , we seal off all pathways of indirect influence, which makes ∂ log p ( Z i | Z = i ) ∂ Θ k = 0 for all Z .

̸

It is worth stressing that p ( Z ) = p ( X | Θ ) p ( Θ ) is a joint distribution over both X and Θ , even though most algorithms operate only on the conditional distribution p ( X | Θ ) . As a consequence, the marginal probability that we place on p ( Θ ) can be a free choice, which is useful for clarifying the assumptions in the model. For example, we will typically assume that synapses do not have any marginalized dependence on one another, i.e. p ( Θ ) = ∏ k p ( Θ k ) .

2.3 Properties of p -locality

Having defined p -locality, we can now talk about its interesting properties, which will give intuitions for how it functions and which will figure prominently in our proofs of algorithms' locality. While any p -local update could also technically be reformulated as a S -local rule, by taking the set S to be the set allowed under p -locality, the conciseness of p -locality and its convenient mathematical properties will prove to be its principal benefits over S -locality. For proofs of these properties, see Appendices A-C. To see the power and generality of p -locality as a definition, we first provide two properties that show how one can immediately make statements about the locality of a learning algorithm by simply inspecting graphs associated with the model. Assume all quantities are defined as in Definition 2.2 and that, for certain properties, the joint density satisfies mild regularity constraints (see Appendix C.1). For the first two properties we further assume that p ( Z ) is strictly positive (see Appendix B). Then we have the following:

̸

Property 2.1. Assume G d is a Directed Acyclic Graph (DAG) for p . If ∂f k ( Z ) ∂ Z i = 0 for Z i that is not a parent, co-parent, or child of Θ k in G d , then f is not p -local.

̸

Property 2.2. Assume that G defines an Undirected Graph (UG) for p . If ∂f k ( Z ) ∂ Z i = 0 for Z i that is not a neighbour of Θ k in G , then f is not p -local.

By these properties, as long as the conditional dependencies of p ( Z ) can be summarized by an UG or DAG (two classes which subsume many modern neural network architectures, see Figure 1 for

̸

examples), we can identify many of the variables disallowed by p -locality. Therefore, for such a network, we can decide whether p -locality conforms to intuitions about biological plausibility as easily as we can for a hand-crafted set of allowed variables under S -locality; for most practical neural network architectures, we will see below that p -locality does behave intuitively. Thus, with p -locality we have a definition of locality that does not require an exhaustive list of the variables available to a synapse, but rather, which relies on the implicit computational structure of the model (i.e. p ( Z ) ).

Property 2.3. For any function b : R N X + N Θ → R N Θ defined such that b k ( Z ) = h k ( f k ( Z ) , g k ( Z )) , where f ( Z ) and g ( Z ) are p -local and h k is differentiable, b ( Z ) is also p -local.

This demonstrates that p -local functions can be arbitrarily combined without the combination losing the p -local property. It also shows that p -locality places no restrictions on the functional form of parameter updates, so long as they are exclusively functions of variables allowed under p -locality.

Property 2.4. For any function f ( · ) : R N X + N Θ → R N Θ , there exists a probability distribution p ( Z ) such that the random variable f ( Z ) with Z ∼ p ( Z ) is p -local.

This is intended as a cautionary note: unless p defines a neural network-like probability distribution, p -locality does not necessarily conform to intuitions about biological locality (see Appendix F for examples). That f ( Z ) is p -local for some probability distribution says nothing about a function f ( Z ) ; p -locality only becomes informative when we narrow our focus to probability distributions with biological relevance.

The following properties will turn out to be extremely important in Section 3: all algorithms that we survey below with provable p -locality properties will involve the score function or the derivative of the unnormalized energy function.

Property 2.5. The derivative of the log joint distribution ∂ log p ( X , Θ ) ∂ Θ is p -local.

Property 2.6. For a probability distribution given by p ( Z ) = 1 Z exp( -E ( Z )) , where Z is a normalizing constant, the expression ∂ ∂ Θ E ( Z ) is p -local.

Property 2.7. If the parameter marginal distribution factorizes as p ( Θ ) = ∏ k p ( Θ k ) , i.e. the parameters are independent from one another, then the score function ∂ log p ( X | Θ ) ∂ Θ is p -local.

Property 2.8. For a mixture distribution p 12 ( Z , γ ) = p 1 ( Z ) γ p 2 ( Z ) 1 -γ p ( γ ) for some binary variable γ ∈ { 0 , 1 } with nonzero probabilities, if f ( Z ) is p 1 -local (or equivalently p 2 -local), then f ( Z ) is p 12 -local.

Intuitively, this feature holds because mixture distributions introduce more dependencies between parameters Θ and random variables X . Therefore, p -locality under a mixture of probability distributions is more permissive than p -locality under any single one of its constituent probability distributions.

2.4 A simple motivating example

To make our definition of p -locality more concrete, we will now explore how it functions for a common network architecture. Consider a simplified linear-nonlinear feedforward neuron model with L layers of neurons r ( l ) ( X = [ r (1) , ..., r ( L ) ] ) connected by synaptic weights W ( l ) ( Θ = [ W 1 , ..., W ( L ) ] ), with additive Gaussian noise at each layer. Conditioned on our feedforward weight matrices and stimuli s , the probability distribution of neural firing rates is given by:

where for notational simplicity we have taken r (0) = s , h ( · ) is a pointwise nonlinearity, and W ( l ) i : corresponds to the i th row of W ( l ) . As discussed above, most algorithms operate on the conditional distribution, p ( X | Θ ) , so we only define a prior over parameters for the purposes of assessing p -locality. To ensure that our joint distribution is a directed graphical model, and to ensure that we have no dependencies between parameters, we will assume that the parameters for the network are all independently distributed, i.e. p ( W ) = ∏ i,j,l p ( W ( l ) ij ) , where W ( l ) ij is the synapse between

a

b

c

postsynaptic neuron i and presynaptic neuron j in layer l . To make our choice of conditional probability distribution more concrete (see Figure 1a), we can see that neural firing rates for this model can be sampled by:

where η ( l ) ∼ N (0 , 1) . This corresponds to an ordinary multilayer perceptron neural network with noise added at every layer. We can use this model to get intuition for how this choice of probability distribution p constrains the variables allowed for an update for a particular parameter W ( l ) ij .

̸

Now, our probability distribution corresponds to a directed graphical model with dependencies given by Eq. 3, so by Property 2.1, we know that variables that are not parents, coparents, or children of W ( l ) ij violate p -locality if included in the parameter update for W ( l ) ij . Figure 1 summarizes the variables allowed under p -locality for such a DAG. W ( l ) ij has no parents under our graphical model, leaving only children and coparents. W ( l ) ij has only one child, r ( l ) i , whose coparents are given by W ( l ) ik s.t. k = j (all other synapses onto r ( l ) i ), and r ( l -1) (all presynaptic neurons).

̸

This shows that updates for synapses in our simple feedforward network cannot depend on neurons that are in layers > l or < l -1 while maintaining p -locality. We also found that dependencies on neurons within the postsynaptic layer l for indices k = i were also not permissible. As a concrete example, by Property 2.7, we found that the score function of p ( X | Θ ) is p -local if the parameters are assumed to be independently distributed. Our example satisfies this criterion. Its score function is given by: ∂ log p ( r | s , W ) ∂ W ( l ) ij = ( r ( l ) i -h ( V ( l ) i ) ) σ 2 h ′ ( V ( l ) i ) r ( l -1) j , where V ( l ) i = W ( l ) i : r ( l -1) . By inspection, this function only requires information about the post- and presynaptic neurons, as well as the summed input to the postsynaptic neuron, V i .

This simple example clearly outlines why p -locality aligns with expected notions of locality. More importantly, the use of p -locality also has the benefit of generalizing cleanly to alternative network architectures. For example, if we were to instead inspect a multilayer recurrent neural network (Figure 1b), we would see that for W ( l ) ij the firing rates r ( l ) i ( t ) and all presynaptic neural firing rates would be permissible for all time steps , because the activities at each timestep are children or coparents of W ( l ) ij . For another classical example of a biologically plausible probability distribution corresponding to an UG (as in Figure 1c.), see Appendix D for a discussion of a linear continuous Boltzmann machine.

2.5 S p -locality

In almost all situations, a given class of plasticity algorithms will produce updates that are almost p -local, in that synapses are required to have access to some small number of global state variables that a biological system could in principle project diffusely throughout a network, but which would not be allowed under generic p -locality (e.g. reward, which is usually produced far downstream of any given synapse in a neural network). For this eventuality, we introduce a notion called S p -locality, which is best considered a hybrid of S -locality and p -locality.

Definition 2.3. Given parameters Θ ∈ R N Θ and random variables X ∈ R N X , concatenated as Z = [ X , Θ ] ⊤ , and a function f ( Z ) : R N X + N Θ → R N Θ , the update function f ( Z ) is S p -local with respect to probability distribution p ( Z ) and some set S = { S k : S k ⊆ Z } if and only if ∀ k f k = h k ( S k , g k ( Z )) , where h k is an arbitrary function and g k ( Z ) is p -local.

The value of this definition is that it negotiates a compromise between the architecture-generality of p -locality, and the flexibility of S -locality. Obviously if we take S k = Z ∀ k , all functions are S -local, and likewise, if we define p sufficiently generally, all functions are p -local (Property 2.4). However, if some class of optimization algorithms operating on a predefined probability distribution p are provably guaranteed to produce parameter updates that are S p -local for some small set S , then we will have obtained a very concise description of the types of information required by that algorithm.

3 Applying S p -locality to normative plasticity models

Given a probability distribution over a neural network states and parameters, p ( X , Θ ) , which could be experimentally motivated or even observed, it is natural to ask which learning algorithms respect locality principles in the way parameter updates are made during learning. Alternatively, if we denote an algorithm which operates on a probability distribution p ( X , Θ ) to output an update f ( Z ) by A ( p ( X , Θ )) = f ( Z ) , where f ( Z ) : R N X + N Θ → R N Θ , we can ask which probability distributions p ( X , Θ ) and variable collections S leave f ( Z ) S p -local. This is one way S p -locality could potentially facilitate experimentally testing biologically plausible learning models.

To demonstrate the utility of our approach, in this section we characterize the S p -locality properties of a wide variety of algorithms that have historically been used to produce normative models of synaptic plasticity, as summarized in Table 1. Though our framework covers many algorithms, here we expand on three prototypical algorithms in detail, focusing on features that are relevant for our discussion of the algorithms' locality properties. More algorithmic details as well as theorem proofs are provided in Appendix E. Schematics depicting biological interpretations for the types of S p -locality discussed below are provided in Figure 2.

3.1 REINFORCE

REINFORCE [13], also known as policy gradient learning, produces reward-modulated Hebbian parameter updates for neural networks [1] similar to the one discussed in Section 2.4. Here we will show that REINFORCE is Rp -local, meaning that it is S p -local where S = { S k = R ∀ k } , which assumes each synapse has access to a global scalar reward signal R that needs to be delivered diffusely to all synapses in a network.

Theorem 3.1. If p ( Θ ) = ∏ k p ( Θ k ) , the REINFORCE estimator given by A R ( p ( R, X | Θ )) is Rp -local.

As an example, consider the network defined by Eq. 3. The REINFORCE update for this network, for a single sample of the network state r and reward R , is given by:

where again V ( l ) i = W ( l ) i : r ( l -1) . This is just the scalar multiplication of the reward signal R with the score function; because the score function is p -local, the full update is Rp -local (see Appendix E for more detail).

3.2 Wake-Sleep (WS)

Whereas REINFORCE has been used to model reinforcement learning in neural networks, the WakeSleep algorithm has been used to model unsupervised learning [22]. It assumes that a neural network has two modes of operation, controlled by a scalar variable γ that determines whether the network is in inference mode ('wake', γ = 1 ) or in generative mode ('sleep,' γ = 0 ). In all, neural activity samples are drawn from the mixture distribution given by p md ( r , γ | W , M ) = p d ( r | W ) γ p m ( r | M ) (1 -γ ) p ( γ ) , where p ( γ ) defines the probability of sampling from the wake phase ( p d ; d corresponds to 'data') or the sleep phase ( p m ; m corresponds to 'model'), and W corresponds to the feedforward weights while M correspond to feedback weights. For conciseness, in all subsequent sections we will denote the mixture distribution p 1 ( Z ) γ p 2 ( Z ) 1 -γ p ( γ ) by Mix ( p 1 ( Z ) , p 2 ( Z )) . As an example, building on the feedforward network defined by Eq. 3 by adding a generative feedback pathway, we have neural dynamics given by:

where h w ( · ) corresponds to the 'wake' nonlinearity, and h s ( · ) corresponds to the 'sleep' nonlinearity. Under this formulation, γ gates, for all neurons , whether activity is driven by the feedforward or feedback pathways; there are several hypotheses for how this could be implemented at a neuronal level, including global neuromodulatory or inhibitory gating of the apical and basal dendrites of pyramidal neurons in the cortex [23] (Fig 2b).

The Wake-Sleep algorithm updates M to fit p m as a generative model of the network's input data, and updates W to fit p d to perform approximate inference with respect to that generative model, in a manner similar to variational autoencoders (V AE) [24, 25]; unlike V AE training with backpropagation, however, the Wake-Sleep algorithm is γp md -local, meaning S = { S k = γ ∀ k } (see Appendix E for the proof).

As a concrete example, the Wake-Sleep parameter update for W ( l ) ij for a single sample of the state r from the network above is given by:

where V ( l ) i = W ( l ) i : r ( l -1) ; a similar update holds for M . Similar to REINFORCE, this parameter update is only the combination of a scalar variable γ and the score function, and is consequently γp md -local (see Appendix E for the proof). Interestingly, Wake-Sleep is not the only algorithm to obey this form of locality: several other normative plasticity models, including Boltzmann machine learning [26], equilibrium propagation [27], and impression learning [23] have essentially the same p -locality properties. However, some of these have additional requirements on the probability distribution that make them less biologically plausible. For example, Boltzmann machine learning is γp md -local only for distributions that can be captured with energy-based models (which typically require weight symmetry).

3.3 Backpropagation (BP) and its approximations

Here we provide a characterization of the locality of the backpropagation algorithm [28], which as we will show using our definitions, is only a local algorithm with biologically implausible assumptions [20]. This demonstration is primarily important because the S p -locality properties of several biologically plausible backpropagation approximations, including feedback alignment [29], weight mirror [30], and Burstprop [31] satisfy a similar notion of S p -locality, but using more biologically realistic assumptions about S (see Table 1 and Appendix E). Derivations for the backpropagation algorithm and its approximations require more stringent network assumptions (here we assume a feedforward multilayer perceptron network as in Section 2.4), because the algorithm itself is not well-defined for arbitrary probabilistic network architectures.

Table 1: Summarizing S p -locality for classical learning algorithms. First column: the optimization algorithm of interest. Second column: the variant of S p -locality that we have proven in Appendix E. Third column: the network architectures required for both the algorithm and our proofs to work. DAG = Directed Acyclic graphical model, MAP = Maximum a posteriori gradient descent dynamics, WTA = winner-take-all circuit, EBM = Energy-based model, MLP = Multilayer perceptron, RNN = recurrent neural network, error-RNN = error-driven recurrent neural network. 'K-Step' refers to the small, finite number of sampling steps required, as opposed to sampling from a steady-state distribution or calculating an equilibrium, which are more time-intensive. Weight symmetry indicates that symmetric recurrent connectivity is required for the original proposed algorithm. Weight transport indicates that symmetric weights are required exclusively for error propagation.

Theorem 3.3. If p ( Θ ) = ∏ k p ( Θ k ) and p ( X | Θ ) is defined by Eq. 3, the BP update for W ( l ) ij with a loss L ( X ) , given by A BP ( p ( X | Θ ) , L ( X )) is e ( l ) i p -local, where e ( l ) i = d L d¯ r i . Similarly, the updates for feedback alignment, weight mirror, and Burstprop are ˆ e ( l ) i p -local, where ˆ e ( l ) i is given by their respective gradient approximations.

This notion of ˆ e ( l ) i p -locality, where S = { S ( l ) ij = ˆ e ( l ) i ∀ i, j, l } , includes more restrictive biological assumptions than the one provided by Rp -locality and γp md -locality, because it relies heavily on the neuron-specific error ˆ e ( l ) i , which must be then accounted for using physiological mechanisms. For our network architecture given by Eq. 3, the functional form of all backpropagation approximations listed in Table 1 is given by:

where ˆ e i is an algorithm-specific approximation of d L d¯ r i . Similar to γp md -local algorithms, recent work has suggested that this error signal could be propagated through networks by a top-down error signal relayed to the apical dendrites of pyramidal neurons [31] (see Fig. 2c). Thus, these algorithms make specific biological predictions, driven by the need for ˆ e ( l ) i to be accessible to each synapse.

4 Discussion

We have constructed a formal definition of locality, S p -locality, that combines set-based and probabilistic graphical model-based formalisms ( S -locality and p -locality, respectively). Our goal was standardizing and operationalizing notions of locality, and we also demonstrated the usefulness of these definitions in characterizing normative plasticity models. Notably, this approach enabled the identification of distinct classes of S p -locality (Table 1) which subsume learning algorithms spanning

more than thirty years of research, from Boltzmann machines [26] to modern incarnations like equilibrium propagation [27]. Importantly, our framework can be applied to any neural network architecture, incorporating networks that are recurrent [16, 32, 23, 27, 26, 33], spiking [34, 12], and multilayer [29, 22, 30] into a single framework. This is thanks, in large part, to the architecture-generality of p -locality, which abstracts away specific details of network models, defining locality instead in terms of statistical dependencies-this makes our framework much more powerful than simpler linguistic descriptions of locality. As we discuss in Appendix F, this abstraction comes at the cost of requiring researchers to specify a distribution, p , that is grounded in biological constraints. We note that for non-biological probability distributions the variables allowed to be 'local' under p -locality can violate standard intuitions; however, for standard neural networks like those discussed in Section 2.4 and Appendix D, p -locality behaves as one would expect, with 'local' learning rules being those that use pre- and postsynaptic activity.

Table 1 organizes existing normative plasticity models into distinct classes. The first locality class, Rp -locality, requires individual synapses to have access to reward information; this class encompasses many reward-modulated Hebbian plasticity models [1]. The second class, p ( m ) -locality, encompasses predictive coding and its variations [33, 35], and aligns with more traditionally Hebbian plasticity updates (often at the cost of unrealistic network architectures; see Table 1). The third class, γp md -locality, requires neural networks to have two distinct modes of operation (e.g. the 'wake' and 'sleep' modes in the Wake-Sleep algorithm) gated by a scalar global variable γ . Thus, γp md -locality requires synapses to have access to the value of γ to compute updates, and subsumes many algorithms that require generative feedback pathways for learning [22, 23]. The final class of algorithms are ˆ e ( l ) i p -local for feedforward neural networks, where ˆ e ( l ) i is a neuron-specific approximate error signal, and includes feedback alignment [29], weight mirror [30], and Burstprop [31], as well as temporal variants of these algorithms [36, 37]. These four categoriesRp -locality, p ( m ) -locality, γp md -locality, and ˆ e ( l ) i -locality-subsume, to our knowledge, almost all existing normative plasticity models.

Because these plasticity models are sometimes only constructed for specific network architectures (which may violate known facts about the brain), we also specify the broadest class of network architectures for which we have successfully derived a model's S p -locality properties (proven in Appendix E). As noted in Appendix F, 'biological plausibility' requires both network architectures and parameter updates to be in line with existing experimental evidence, while S p -locality provides a formal framework for assessing only the latter of these two factors. We therefore stress that S p -locality properties are only relevant for neuroscience if they can be determined for biologically plausible network architectures. As a consequence, classes of network architectures that encompass a broad range of biologically plausible network models (such as DAGs) are desirable, whereas algorithms that are only viable for implausible architectures (e.g. those that imply weight symmetry or weight transport) are undesirable.

Interestingly, the distinct classes of locality also delineate distinct sets of experimentally testable predictions for plasticity. Critically, S p -locality abstracts away details that are not important for testing predictions, and helps identify important features of learning algorithms. For example, the REINFORCE (Eq. 6) and Wake-Sleep (Eq. 8) algorithms are equivalent with respect to the ' p ' in S p -locality, which tells us that the set of allowed local variables under p -locality itself (e.g. pre- and post-synaptic information) are not the best targets for experimental testing between these algorithms, and instead the focus must be on the variables included in ' S ' (a reward signal R versus a clamping variable γ that switches the network between different modes of synaptic plasticity and neural activity). Thus, by identifying whether plasticity is modulated by reward, neuron-specific error teaching signals, transitions in network dynamics, or none of the above, one may narrow the space of possible candidate plasticity models down to Rp -, ˆ e ( l ) i -, γp md -, or p ( m ) -locality, respectively. These distinct phenomena are necessary features of the learning algorithms in question for any neural network architecture used and are consequently good targets for experimental testing. In contrast, other features of parameter updates are more about specific architectural choices, e.g. switching from a rate-based network [16] to spiking [15], from feedforward [22] to recurrent [32], or singleto multi-compartment [38]. Thus, beyond clarifying and categorizing the locality properties of normative plasticity models, our hope is that our framework will also support experimental efforts to differentiate between such models, helping the field to focus in on the most important predictions to test experimentally.

5 Acknowledgments

BR was supported by NSERC (Discovery Grant: RGPIN-2020-05105; Discovery Accelerator Supplement: RGPAS-2020-00031; Arthur B. McDonald Fellowship: 566355-2022), CIFAR (Canada AI Chair; Learning in Machine and Brains Fellowship), the Canada Research Chair in Neural Computations and Interfacing. GL was supported by NSERC (Discovery Grant: RGPIN-201804821), CIFAR (Canada AI Chair), and a Canada Research Chair in Neural Computations and Interfacing. CB and GL acknowledge support from a Simon's Collaboration on the Global Brain Pilot Award. EW was supported by an NSERC CGS-D scholarship. CS was supported by the National Science Foundation under NSF Award No. 1922658.

References

Nicolas Frémaux and Wulfram Gerstner. Neuromodulated spike-timing-dependent plasticity, and theory of three-factor learning rules. Frontiers in neural circuits , 9:85, 2016.

Blake A Richards and Timothy P Lillicrap. Dendritic solutions to the credit assignment problem. Current opinion in neurobiology , 54:28-36, 2019.

Claudia Clopath, Tim P Vogels, Robert C Froemke, and Henning Sprekeler. Receptive field formation by interacting excitatory and inhibitory synaptic plasticity. BioRxiv , page 066589, 2016.

Yuhan Helena Liu, Stephen Smith, Stefan Mihalas, Eric Shea-Brown, and Uygar Sümbül. Cell-typespecific neuromodulation guides synaptic credit assignment in a spiking neural network. Proceedings of the National Academy of Sciences , 118(51):e2111821118, 2021.

Matt Udakis, Victor Pedrosa, Sophie EL Chamberlain, Claudia Clopath, and Jack R Mellor. Interneuronspecific plasticity at parvalbumin and somatostatin inhibitory synapses onto CA1 pyramidal neurons shapes hippocampal output. Nature communications , 11(1):4395, 2020.

Dimitar Kostadinov, Maxime Beau, Marta Blanco-Pozo, and Michael Häusser. Predictive and reactive reward signals conveyed by climbing fiber inputs to cerebellar Purkinje cells. Nature neuroscience , 22(6):950-962, 2019.

Aparna Suvrathan and Jennifer L Raymond. Depressed by learning-heterogeneity of the plasticity rules at parallel fiber synapses onto Purkinje cells. The Cerebellum , 17:747-755, 2018.

Wei Guo, Blaise Robert, and Daniel B Polley. The cholinergic basal forebrain links auditory stimuli with delayed reinforcement to support learning. Neuron , 103(6):1164-1177, 2019.

José Fernando Maya Vetencourt, Ettore Tiraboschi, Maria Spolidoro, Eero Castrén, and Lamberto Maffei. Serotonin triggers a transient epigenetic mechanism that reinstates adult visual cortex plasticity in rats. European Journal of Neuroscience , 33(1):49-57, 2011.

Katie C Bittner, Aaron D Milstein, Christine Grienberger, Sandro Romani, and Jeffrey C Magee. Behavioral time scale synaptic plasticity underlies CA1 place fields. Science , 357(6355):1033-1036, 2017.

Ila R Fiete, Michale S Fee, and H Sebastian Seung. Model of birdsong learning based on gradient estimation by dynamic perturbation of neural conductances. Journal of neurophysiology , 98(4):2038-2057, 2007.

Robert Legenstein, Dejan Pecevski, and Wolfgang Maass. A learning theory for reward-modulated spiketiming-dependent plasticity with application to biofeedback. PLoS computational biology , 4(10):e1000180, 2008.

Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Reinforcement learning , pages 5-32, 1992.

Roman Pogodin and Peter Latham. Kernelized information bottleneck leads to biologically plausible 3-factor Hebbian learning in deep networks. Advances in Neural Information Processing Systems , 33:72967307, 2020.

Nicolas Frémaux, Henning Sprekeler, and Wulfram Gerstner. Reinforcement learning using a continuous time actor-critic framework with spiking neurons. PLoS computational biology , 9(4):e1003024, 2013.

Thomas Miconi. Biologically plausible learning in recurrent neural networks reproduces neural dynamics observed during cognitive tasks. Elife , 6:e20899, 2017.

Robert Urbanczik and Walter Senn. Learning by the dendritic prediction of somatic spiking. Neuron , 81(3):521-528, 2014.

Jordan Guerguiev, Timothy P Lillicrap, and Blake A Richards. Towards deep learning with segregated dendrites. Elife , 6:e22901, 2017.

João Sacramento, Rui Ponte Costa, Yoshua Bengio, and Walter Senn. Dendritic cortical microcircuits approximate the backpropagation algorithm. Advances in neural information processing systems , 31, 2018.

Timothy P Lillicrap, Adam Santoro, Luke Marris, Colin J Akerman, and Geoffrey Hinton. Backpropagation and the brain. Nature Reviews Neuroscience , 21(6):335-346, 2020.

Owen Marschall, Kyunghyun Cho, and Cristina Savin. A unified framework of online learning algorithms for training recurrent neural networks. The Journal of Machine Learning Research , 21(1):5320-5353, 2020.

Geoffrey E Hinton, Peter Dayan, Brendan J Frey, and Radford M Neal. The "wake-sleep" algorithm for unsupervised neural networks. Science , 268(5214):1158-1161, 1995.

Colin Bredenberg, Benjamin Lyo, Eero Simoncelli, and Cristina Savin. Impression learning: Online representation learning with synaptic plasticity. Advances in Neural Information Processing Systems , 34:11717-11729, 2021.

Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 , 2013.

Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic backpropagation and approximate inference in deep generative models. In International conference on machine learning , pages 1278-1286. PMLR, 2014.

David H Ackley, Geoffrey E Hinton, and Terrence J Sejnowski. A learning algorithm for Boltzmann machines. Cognitive science , 9(1):147-169, 1985.

Benjamin Scellier and Yoshua Bengio. Equilibrium propagation: Bridging the gap between energy-based models and backpropagation. Frontiers in computational neuroscience , 11:24, 2017.

Yann LeCun, Bernhard Boser, John S Denker, Donnie Henderson, Richard E Howard, Wayne Hubbard, and Lawrence D Jackel. Backpropagation applied to handwritten zip code recognition. Neural computation , 1(4):541-551, 1989.

Timothy P Lillicrap, Daniel Cownden, Douglas B Tweed, and Colin J Akerman. Random synaptic feedback weights support error backpropagation for deep learning. Nature communications , 7(1):13276, 2016.

Mohamed Akrout, Collin Wilson, Peter Humphreys, Timothy Lillicrap, and Douglas B Tweed. Deep learning without weight transport. Advances in neural information processing systems , 32, 2019.

Alexandre Payeur, Jordan Guerguiev, Friedemann Zenke, Blake A Richards, and Richard Naud. Burstdependent synaptic plasticity can coordinate learning in hierarchical circuits. Nature neuroscience , 24(7):1010-1019, 2021.

G Hinton, Peter Dayan, Ava To, and Radford Neal. The Helmholtz machine through time. In ICANN-95 , pages 483-490, 1995.

Rajesh PN Rao and Dana H Ballard. Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nature neuroscience , 2(1):79-87, 1999.

Peter Dayan and Geoffrey E Hinton. Varieties of Helmholtz machine. Neural Networks , 9(8):1385-1403, 1996.

Beren Millidge, Anil Seth, and Christopher L Buckley. Predictive coding: a theoretical and experimental review. arXiv preprint arXiv:2107.12979 , 2021.

James M Murray. Local online learning in recurrent networks with random feedback. Elife , 8:e43299, 2019.

Guillaume Bellec, Franz Scherr, Anand Subramoney, Elias Hajek, Darjan Salaj, Robert Legenstein, and Wolfgang Maass. A solution to the learning dilemma for recurrent networks of spiking neurons. Nature communications , 11(1):3625, 2020.

Panayiota Poirazi, Terrence Brannon, and Bartlett W Mel. Pyramidal neuron as two-layer neural network. Neuron , 37(6):989-999, 2003.

Judea Pearl. Probabilistic reasoning in intelligent systems: networks of plausible inference . Morgan kaufmann, 1988.

Steffen L Lauritzen. Elements of graphical models. Lectures from the XXXVIth International Probability Summer School in St-Flour, France. http://www. stats. ox. ac. uk/steffen , 2011.

Richard E Neapolitan et al. Learning bayesian networks , volume 38. Pearson Prentice Hall Upper Saddle River, 2004.

Christopher M Bishop and Nasser M Nasrabadi. Pattern recognition and machine learning , volume 4. Springer, 2006.

Sam Roweis and Zoubin Ghahramani. A unifying review of linear Gaussian models. Neural computation , 11(2):305-345, 1999.

Arthur P Dempster, Nan M Laird, and Donald B Rubin. Maximum likelihood from incomplete data via the EM algorithm. Journal of the royal statistical society: series B (methodological) , 39(1):1-22, 1977.

Geoffrey J McLachlan and Thriyambakam Krishnan. The EM algorithm and extensions . John Wiley & Sons, 2007.

Ezekiel Williams, Colin Bredenberg, and Guillaume Lajoie. Flexible phase dynamics for bio-plausible contrastive learning. In Proceedings of the 40th International Conference on Machine Learning , volume 202 of Proceedings of Machine Learning Research , pages 37042-37065. PMLR, 23-29 Jul 2023.

Shiro Ikeda, Shun-ichi Amari, and Hiroyuki Nakahara. Convergence of the wake-sleep algorithm. Advances in neural information processing systems , 11, 1998.

Bernhard Nessler, Michael Pfeiffer, Lars Buesing, and Wolfgang Maass. Bayesian computation emerges in generic cortical microcircuits through spike-timing-dependent plasticity. PLoS computational biology , 9(4):e1003037, 2013.

Ronald J Williams and David Zipser. A learning algorithm for continually running fully recurrent neural networks. Neural computation , 1(2):270-280, 1989.

Aditya Gilra and Wulfram Gerstner. Predicting non-linear dynamics by stable local learning in a recurrent spiking neural network. Elife , 6:e28295, 2017.

Thomas L O'Donohue, William R Millington, Gail E Handelmann, Patricia C Contreras, and Bibie M Chronwall. On the 50th anniversary of Dale's law: multiple neurotransmitter neurons. Trends in Pharmacological Sciences , 6:305-308, 1985.

Kevin Ellis, Catherine Wong, Maxwell Nye, Mathias Sable-Meyer, Luc Cary, Lucas Morales, Luke Hewitt, Armando Solar-Lezama, and Joshua B Tenenbaum. Dreamcoder: Growing generalizable, interpretable knowledge with wake-sleep bayesian program learning. arXiv preprint arXiv:2006.08381 , 2020.

Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 , 2013.

A Defining Markov locality and relating it to p -locality

To gain intuition for how p -locality functions, we will introduce another notion of locality, called Markov locality , which will use the language of Markov blankets. We will prove that under relatively relaxed conditions p -locality and Markov locality are equivalent. This will allow us to relate the notion of locality to various graph structures commonly used to represent probability distributions, and will be a key step in proving Properties 2.1 and 2.2.

We start by defining the Markov boundary, M ( X,S ) , of a random variable X contained in a set of random variables S , as a minimal set such that p ( X | S ) = p ( X |M ( X,S )) . The Markov boundary defines a minimal set of variables such that, conditioned on these variables, conditioning on no additional random variables in S changes the probability of X [39]. Similarly, we define the Markov blanket, M ( X,S ) for X in S as any set of variables such that conditioning on M ( X,S ) , makes X conditionally independent from all other variables [39]. In this way, the Markov boundary is a Markov blanket but not all blankets are boundaries.

Definition A.1. Markov locality: Given probability distribution p ( Z ) and function f : R N X + N Θ → R N Θ , the update function f ( Z ) is Markov-local with respect to the distribution p over Z if and only if ∀ k :

̸

Markov locality requires that the set of variables used in the parameter update f k ( Z ) is a subset of the Markov boundary of the parameter itself. A Markov boundary can be thought of as the set of variables that 'locally' communicate with the parameter Θ k , thus providing a natural measure of locality.

Importantly, for Markov-locality to be of use, we would like the Markov boundaries of random variables in the model of interest to be unique. Without this requirement there will be ambiguity, for a given p , in terms of which updates are considered local and which are not. To guarantee this, we ask that the conditional independence relationships implied by p satisfy four properties, commonly referred to as graphoid properties [39, 40]. A sufficient condition for these to hold is that the distribution have a strictly positive density (see Appendix for more details B). With this, and some mild regularity assumptions, we can prove the following equivalence between Markov locality and p -locality:

Theorem A.1. Assume all quantities are as in A.1, that the conditional independence relationships implied by p ( Z ) satisfy the four graphoid properties given in Section B, and that mild regularity assumptions are satisfied by the joint distribution (see Section C.1). Then Equation 2 holds if and only if Equation A.1 also holds.

Proof. This proof relies on Lemma A.1, proved below.

We wish to prove Eq. 2 ⇐⇒ Eq. A.1. It suffices to show the following:

̸

̸

Using the contrapositive for the left and right implications separately shows that Equation A.2 is equivalent to

̸

which means that it suffices to prove Equation A.3 for the proof. Observe that

̸

̸

which follows from the regularity assumptions. From here, the proof follows by Lemma A.1.

Lemma A.1. Let X , Θ , Z , k and i , and p be defined as in Theorem A.1. Then

̸

̸

̸

by the chain rule. By applying the fundamental theorem of calculus to this derivative, which we can do by the assumption of differentiability on R , we find that p ( Z i | Z = i ) is constant w.r.t. Z k on Ω so that

̸

̸

̸

̸

̸

̸

where we have also used that a probability distribution integrates to 1 , and that p ( Z k | Z = { i,k } ) will be equal to zero outside Ω . Because Z ∈ Ω is arbitrary, from the above Z i is independent of Z k given the other random variables in Z . Using Lemma B.1 (which we can do by assumption of the graphoid properties), we see that if ∂ log p ( Z i | Z = i ) ∂ Z k = 0 ∀ Z ∈ Ω , by Eq. A.6 - A.8, Z i / ∈ M ( Z k , Z ) . Conversely, if we start with the assumption that Z i / ∈ M ( Z k , Z ) , we immediately get Z k / ∈ M ( Z i , Z ) , by Lemma B.1, and see that p ( Z i | Z = i ) must not be a function of Z k for all Z ; thus, the derivative w.r.t. Z k is equal to zero for Z ∈ Ω . Applying Equation A.6 completes the proof.

̸

B Notes on Probabilistic Graphical Models

In this section we compile several properties, definitions, and results on Markov boundaries and Probabilistic Graphical Models (PGMs) that underlie Theorem A.1, and Properties 2.1 and 2.2. We begin by setting up notation. Let us assume that we have a joint probability distribution, P , over a set of random variables S , and that W , X , Y , Z ⊂ S , and U , V , T ∈ S . We use X ⊥ ⊥ Y | Z to mean that the set of random variables X is independent of the set Y given set Z , and assume that the reader is familiar with the notion of a directed graph, an undirected graph, and graph separation. If a set X contains only a single random variable U then we abuse notation and write U in place of X .

The following four properties-known as the graphoid properties (or axioms-see e.g. [39, 40])-are useful for getting well-behaved Markov boundaries and in assigning graphical representation to probability distributions:

Definition B.1. Pseudo-graphoid properties:

Symmetry: X ⊥ ⊥ Y | Z = ⇒ Y ⊥ ⊥ X | Z

Decomposition: X ⊥ ⊥ Y, W | Z = ⇒ X ⊥ ⊥ Y | Z & X ⊥ ⊥ W | Z

Weak union: X ⊥ ⊥ Y, W | Z = ⇒ X ⊥ ⊥ Y | W,Z

Intersection: X ⊥ ⊥ Y | Z, W & X ⊥ ⊥ W | Z, Y = ⇒ X ⊥ ⊥ Y, W | Z

Proof. First, observe that

̸

̸

Importantly, it is known that these properties are satisfied when we have a density p that is strictly positive w.r.t. to its base product measure [40]. Here, measure is used in a measure theoretic sense; e.g. we have assumed throughout the paper that the base measure is simply a product of a multidimensional Lebesgue measure over R k 1 , for some k 1 , and a counting measure over N k 2 or some subset of N k 2 , for some k 2 . Roughly speaking, this positivity property means that there are no purely deterministic relationships between variables.

The key result that we use to guarantee that the Markov boundaries we discuss in the paper are well-defined is given in [39]. We state a paraphrased and shortened version below for completeness:

Theorem B.1. Theorem 4, Chapter 3 in [39]: every P with conditional independence relations satisfying the four pseudo-graphoid properties has a unique Markov boundary for each X .

We now add two more simple results on Markov boundaries, used in the proof of Theorem A.1:

Lemma B.1. If P has conditional independence relations satisfying the four pseudo-graphoid properties we have:

for every U, V ∈ S , U ∈ M ( V, S ) ⇐⇒ V ∈ M ( U, S )

for every U ∈ S , M ( U, S ) is contained in every Markov blanket of U .

These follow simply from the graphoid properties so we omit the proof.

Lastly, we make specific what we mean when we say that a graph is an undirected or directed graphical model for a distribution.

Definition B.2. Let G be an undirected graph where each node corresponds to a random variable in S . We say that G is an Undirected Graph (UG) for P if whenever X and Y are separated by Z in G , X ⊥ ⊥ Y | Z is true under P . Note that this corresponds to the notion of I-map in [39].

Definition B.3. Let G d be a directed graph with each vertex corresponding to a random variable in S . We say that G d is a Directed Graph for P if the variable under P corresponding to any node in the graph is conditionally independent of all variables corresponding to nodes that are non-descendants given the variables corresponding to parents. This is equivalent to G d satisfying the Markov condition described in Definition 1.9 of [41].

C Proofs for p -locality properties

For the first two properties we assume the requirements of Theorem A.1 are satisfied. For all properties except 2.3, 2.4, and 2.8 we assume p satisfies mild regularity constraints (see Section C.1).

̸

Property 2.1 Assume G d is a Directed Acyclic Graph (DAG) for p . If ∂f k ( Z ) ∂ Z i = 0 for Z i that is not a parent, co-parent, or child of Θ k in G d , then f is not p -local.

Proof. By Theorem A.1 we get that ∂f k ( Z ) ∂ Z i can only be non-zero on the unique Markov boundary of Z i if it is p -local. By the definition of a DAG, the parents, co-parents, and children of Z i form a Markov blanket for it (see e.g. [41] Th. 2.13). By Lemma B.1 the boundary is included in all Markov blankets so ∂f k ( Z ) ∂ Z i can only be non-zero on some subset of the parents, co-parents, and children of Z i .

̸

Property 2.2 Assume that G defines an Undirected Graph (UG) for p . If ∂f k ( Z ) ∂ Z i = 0 for Z i that is not a neighbour of Θ k in G , then f is not p -local.

Proof. As above, by Theorem A.1 we get that ∂f k ( Z ) ∂ Z i can only be non-zero on the unique Markov boundary of Z i if it is p -local. A UG for a distribution is an I-map for it, and conditioning on the neighbours in an I-map renders a node independent from the other nodes in the graph by definitionthus the neighbours form a Markov blanket. By Lemma B.1 the Markov boundary is included in every blanket so ∂f k ( Z ) ∂ Z i can only be non-zero on some subset of the neighbours in the UG.

Property 2.3 For any function b ( Z ) : R N X + N Θ → R N Θ defined such that b k ( Z ) = h k ( f k ( Z ) , g k ( Z )) , where f and g are p -local and h k is differentiable, b ( Z ) is also p -local.

̸

̸

Proof. Suppose that ∂b k ( Z ) ∂ X i = 0 . We need to show that E p [ ( ∂ log p ( Z i | Z = i ) ∂ Θ k ) 2 ] = 0 . Knowing that ∂b k ( Z ) ∂ X i = 0 , we have:

̸

̸

̸

̸

This implies that either ∂f k ( Z ) ∂ Z i = 0 or ∂g k ( Z ) ∂ Z i = 0 (or both). No matter which is true, by virtue of the p -locality of f and g , we have the consequence:

̸

̸

which concludes our proof. This demonstrates that p -local functions can be more or less arbitrarily combined without the combination losing the p -local property.

Property 2.4 For any function f ( · ) : R N X + N Θ → R N Θ , there exists a probability distribution p ( Z ) such that the random variable f ( Z ) with Z ∼ p ( Z ) is p -local.

̸

Proof. We can prove this property by construction. Take p ( { X i : ∂f ( X ) ∂ X i = 0 }| Θ ) = N ( Θ T Θ , I ) , i.e. the distribution of every variable contained within f has mean parameter dependence on all Θ variables. The probability distributions for all other variables Z are otherwise unconstrained. Then for all i such that ∂f ( X ) ∂ X i = 0 , we have:

̸

̸

̸

Property 2.5 The derivative of the log joint distribution ∂ log p ( X , Θ ) ∂ Θ is p -local.

Here, it's more useful to work with the equivalent (contrapositive) requirement for p -locality, i.e., we need to show ∀ k, i :

̸

̸

̸

̸

Proof. First, we see that:

̸

where the first implication follows from the fact that the Fisher Information integral is effectively a weighted sum of elements, each of which is ≥ 0 . If the function on the right were nonzero for some Z , then the Fisher Information would also be nonzero. Assuming that log p has differentiable partial derivatives, we can interchange the order of differentiation, giving:

̸

̸

̸

which concludes the proof.

Property 2.6 For a probability distribution given by p ( Z ) = 1 Z exp( -E ( Z )) , the expression ∂ ∂ Θ E ( Z ) is p -local.

Proof. The proof is almost identical to the proof for Property 2.5. From Property 2.5, we have that:

̸

Using our definition of p , we have:

which concludes the proof.

Property 2.7 If the marginal parameter distribution factorizes as p ( Θ ) = ∏ k p ( Θ k ) , i.e. the parameters are independent from one another, then the score function ∂ log p ( X | Θ ) ∂ Θ is p -local.

Proof. Again, we make heavy use of Property 2.5, which states:

̸

̸

It is important to note that the left-hand equation only holds true if Z i = Θ k : under p -locality, an update equation for parameter Θ k can always include its own value. So for the remainder of the proof we will assume that Z i = Θ k . Now, log( p ( X | Θ )) = log p ( Z ) -log p ( Θ ) . We also have:

̸

where for the last equality we have used the assumption that Z i = Θ k . These two equations collectively imply:

which concludes the proof.

̸

Property 2.8 For a mixture distribution p 12 ( Z , γ ) = p 1 ( Z ) γ p 2 ( Z ) 1 -γ p ( γ ) for some binary variable γ ∈ { 0 , 1 } with nonzero probabilities, if f ( Z ) is p 1 -local (or equivalently p 2 -local), then f ( Z ) is p 12 -local.

Proof. We again work with the contrapositive definition of p -locality, observing that:

̸

̸

̸

where the third implication follows from the fact that if the sum of two nonnegative quantities is zero, then both quantities are zero, and the final implication holds from the p 1 -locality of f ( Z ) .

C.1 Regularity of Joint distribution

For several of these properties we enforce mild regularity constraints on the density. This is because we want the integral of the squared score being equal to zero to imply that the score itself is equal to zero. A sufficient condition for this is that the joint density function and partial derivatives w.r.t. Θ k ∀ k are, for every fixed value of Z 's discrete elements, continuous functions of Z 's continuous elements.

D Locality for a linear continuous Boltzmann machine

Consider the following example of a simplified linear recurrent neuron model with synaptic weight matrices W . The joint distribution is given by:

where W is assumed to be a symmetric matrix. To see why this probability distribution is relevant for neuroscience, we first note that E is a linear, continuous analog of the Hopfield energy function, which is also used for discrete-valued Boltzmann machines. There are two critical differences between this probability distribution and the linear feedforward network explored in Section 2.4: first, this distribution corresponds to the undirected graphical model shown in Figure 1c, as opposed to the DAG shown in Figure 1a; second, the marginal distribution is not a free parameter that we can choose with convenient factorization properties if we want our joint distribution to give us a version of p -locality that corresponds with our concept of biological locality. For undirected graphical models, one typically is required to define the joint distribution first, and compute conditional distributions explicitly through Bayes theorem or approximate through some form of MCMC sampling. In our case, we can see that p ( r | W ) corresponds to the steady-state distribution of a stochastic differential equation using E to perform Langevin sampling:

where here B i ( t ) corresponds to uncorrelated Brownian noise injected into the system. These stochastic sampling dynamics correspond to a noisy linear recurrent network. Therefore, p ( r | W ) corresponds to the steady-state stimulus response distribution of a linear recurrent network.

̸

Let's ask: for which neural indices k can we have ∂ ∂ r k f W ij ( r ) = 0 so that the function f is still p -local? For f to remain p -local, we would need E p [ ( ∂ log p ( r k | r = k , W ) ∂ W ij ) 2 ] = 0 . For the definition of p -locality to conform to our intuitions about biological locality, we would expect the only allowable variables to be the pre- and postsynaptic neurons r i and r j -we will show that including any other variable will violate p -locality. To see why, suppose k = i, j . Note that we can decompose E as:

̸

̸

̸

̸

̸

̸

Under this decomposition, E k has no dependency on W ij , and E = k has no dependency on r k . Now we're in a position to demonstrate that for any choice of k such that k = i, j , f W ij ( r ) cannot be p -local.

̸

̸

̸

̸

̸

̸

̸

̸

Because the conditional distribution has no dependency on W ij , then the Fisher Information is also 0, which concludes the demonstration: f W ij is not p -local if it is a function of r k for k = i, j . Of course, this decomposition of E = E k + E = k would not be possible if k = i or j . To summarize, for our simple example, any parameter update for W ij that depends on the activity of any neuron r k that is not the pre- or postsynaptic neuron ( k = i, j ) cannot be p -local. Alternatively, since this is an undirected graphical model, we can also inspect its corresponding graph (summarized in Figure 1c.). To verify that the graph in Figure 1c. corresponds to our network, observe that our probability

̸

̸

̸

̸

distribution factorizes according the cliques of the graph [42] as follows:

Looking at the graph, we can verify by inspection that the only neighbors of W ij are r i and r j , which confirms our detailed analysis by Property 2.2.

E Proofs of p -locality properties of normative plasticity algorithms

E.1 REINFORCE

Theorem E.1. If p ( Θ ) = ∏ k p ( Θ k ) , the REINFORCE estimator given by A R ( p ( R, X | Θ )) is Rp -local.

Proof. The REINFORCE derivation proceeds as follows: suppose that we have some probabilistic formulation of a neural network and incoming sensory stimuli p ( X | Θ ) and some probabilistic reward function p ( R | X ) dependent on the stimuli and neural responses. We want to maximize expected reward:

If we want to modify our parameters Θ in order to improve performance, we take steps in an approximation of the direction of the gradient of the objective E [ R ] .

̸

where in this last step we have employed a Monte Carlo approximation of the expectation, where R ( k ) and X ( k ) ∼ p ( R, X ) . This update function: f ( R, Z ) = R × ∂ ∂ Θ log p ( X | Θ ) is not p -local because we have ∂f ( R, Z ) /∂R = 0 , while ∂ ∂ Θ p ( R | X ) = 0 . However, as we know, this update is the product of a score function with a marginal parameter distribution that we have assumed factorizes, which we know to be p -local by Property 2.7, with a scalar reward R . In this case, one could postulate that reward information is projected broadly to many synapses in the neural network via a neuromodulatory pathway (Figure 2a). We see that f ( R, X ) = h ( R,g ( X )) if we take h ( a, b ) = a × b and g ( X ) = ∂ ∂ Θ log p ( X | Θ ) ; we further see that g is p -local, and hence f ( R, X ) is by Definition 2.3 Rp -local.

This might seem contrived, because any function is S p -local for some sufficiently broad choice of S . However, we have shown here that the REINFORCE algorithm is Rp -local for any choice of p with a marginal parameter distribution that factorizes (an easy constraint to satisfy for directed graphical model architectures). This means that we can make any of a huge variety of neural network or probabilistic model choices and still have the REINFORCE algorithm obey the same notion of locality, without having to modify our definition post-hoc.

E.2 Maximum Likelihood Estimation (MLE)

MLE is a highly popular machine learning method and the fundamental basis for several subsequent normative plasticity algorithms. This algorithm involves fitting a model, p m ( X | Θ ) , to an empirical data distribution, p d ( X ) .

Theorem E.2. If p ( Θ ) = ∏ k p ( Θ k ) , the MLE update given by A MLE ( p m ( X | Θ ) , p d ( X )) is p m -local.

Proof. We proceed by first deriving the MLE update function. The objective function for maximum likelihood estimation is given by the KL divergence between an empirical data distribution, p d ( X ) and a probabilistic model of the data p m ( X | Θ ) . We have:

We want to minimize this objective function, which we do by gradient descent:

where X k ∼ p d ( X ) , and in the last approximate equality we have used a Monte Carlo sampling integral approximation. This update exclusively contains the score function of p m , so by Property 2.7, the update is p m -local.

E.3 Generalized EM (GEM)

MLE estimation runs into difficulties when attempting to fit latent variable models, (e.g. when p m ( X o ) = ∫ p m ( X o , X h ) d X h ), where X h are latent variables within the model distribution that 'explain' observed data X o . Latent variable models are extraordinarily powerful, and appear in computational neuroscience in a variety of forms, including but not limited to factor analysis, hidden Markov models, and Kalman Filters [43]; for these models, we will take X = [ X o , X h ] . Instead of performing explicit MLE, when fitting latent variable models one usually resorts to some variant of the Expectation-Maximization (EM) algorithm [44]. Here, we show that a particular variant of the EM algorithm, called Generalized EM (GEM) [45], is p m -local in the same way as MLE.

GEM gains computational benefits by substituting (by any of a variety of methods) an approximate posterior distribution p d ( X h | X o ) for the true, but typically intractable, model posterior p m ( X h | X o ) via minimizing a variational free energy [45]. However, GEM is not just a convenient model-fitting algorithm: in subsequent sections, we will show that the p m -locality of GEM explains why several popular normative plasticity algorithms produce biologically plausible updates.

Theorem E.3. If p ( Θ ) = ∏ k p ( Θ k ) , the GEM update given by A GEM ( p m ( X | Θ ) , p d ( X )) is p m -local.

Proof. Rather than minimize KL[ p d ( X o ) || p m ( X o | Θ )] , the GEM algorithm minimizes an upper bound (the variational free energy). Taking X = [ X o , X h ] :

where the inequality follows from the positivity of the KL divergence. Here, p d ( X h | X ) is an approximate inference distribution. Different choices of how this distribution is selected/optimized can produce very different learning algorithms, with varying degrees of biological plausibility. Obviously, the loss is minimized with respect to p d ( X h | X ) if p d ( X h | X ) = p m ( X h | X , Θ 0 ) (where Θ 0 = Θ prior to optimization wrt Θ ). This choice corresponds to GEM [45]. For now, we will not concern ourselves with how p d ( X h | X ) is selected-instead, we will focus on the locality properties of gradient updates of this loss with respect to Θ m .

Having packaged hidden and observed variables together ( X = [ X o , X h ] ), our derivation proceeds exactly the same as for MLE:

where as with the MLE update, X k ∼ p m ( X | Θ ) . This update is p m -local for the same reason that the MLE update is.

E.4 Predictive Coding (PC)

As an additional note, if one takes the approximate posterior p d ( X h | X o ) to be given by:

where δ ( · ) indicates a Dirac delta distribution and ¯ X h ( X o ) indicates a set of observation-dependent mean parameters, then we recover the predictive coding family of algorithms [35]. Typically, in this context for a given observed stimulus X o , p d ( X h | X o ) ∼ δ ( ¯ X h ( X o )) is estimated by reparameterization and gradient descent with respect to ¯ X h ( X o ) (a mean parameter that is observation-dependent), which-for clever choices of p m -loosely resembles the dynamics of a recurrent neural network relaxing to a stimulus-conditioned equilibrium state [33]. After estimating ¯ X h ( X o ) , parameters Θ are updated as in GEM. Therefore, the derivation above also applies to predictive coding algorithms, which are consequently also p m -local.

Theorem E.4. If p ( Θ ) = ∏ k p ( Θ k ) , the PC update given by A PC ( p m ( X | Θ ) , p d ( X )) is p m -local.

E.5 Wake-Sleep

Unlike the previous three examples, which only require sampling from the p d distribution and only calculate parameter updates according to the p m distribution, the Wake-Sleep algorithm parameterizes both distributions and jointly samples from a mixture of the two distributions across its 'wake' and 'sleep' phases. As we will see, this will mean that the Wake-Sleep algorithm will end up being γp md -local, where γ is the binary variable that controls whether the system is in its 'wake' or 'sleep' phase.

Theorem E.5. If p ( Θ , Θ ( d ) ) = ( ∏ k p ( Θ k ) )( ∏ k p ( Θ ( d ) k ) ) , the Wake-Sleep estimator given by A WS ( p m ( X | Θ ) , p d ( X | Θ ( d ) )) is γp md -local, where p md = Mix ( p m ( X | Θ ) , p d ( X | Θ ( d ) ) ) p ( Θ , Θ ( d ) ) .

Proof. Our updates use a similar loss to the GEM algorithm, namely we take:

where ∆ Θ WS is given by:

̸

Here, γ k , X ∼ p md ( X , γ | Θ , Θ ( d ) ) , whereas for GEM, we sampled only from p d . Because each term of this update is 0 if γ k = 1 , this update is still an unbiased estimate of the gradient [46] and is effectively equivalent to the GEM update, except that it allows the system to alternate between sampling from p m and p d . This alternation is useful because it will allow also calculating parameter updates for ∆ Θ ( d ) . For Θ ( d ) , we optimize the reverse KL-divergence 4 ; by a directly analogous derivation to Eq. E.12, the update is given by:

Now, these updates contain the score function for both p m and p d , as well as the scalar mixture variable γ . As a consequence, both updates are γp md -local, via Properties 2.7 and 2.8 (slightly more precisely, ∆ Θ is γp m -local, and ∆ Θ ( d ) is γp d -local).

E.6 Impression Learning (IL)

The impression learning parameter update [23] is closely related to the Wake-Sleep parameter update, and is consequently also γp md -local. What distinguishes IL from WS is the use of rapid alternations in the gating signal γ t within a single trial with T time steps. Here, X = [ X 0 , ..., X T ] , γ = [ γ 0 , ..., γ t ] and p md ( X | γ, Θ ) = ∏ T t =0 p d ( X t | X t -1 , Θ ) γ t p m ( X t | X t -1 , Θ ) 1 -γ is a mixture distribution in which γ t alternates between 0 and 1 , sampling from either p m or p d at the time step t , respectively.

Theorem E.6. If p ( Θ ) = ∏ k p ( Θ k ) , the impression learning estimator given by A IL ( p m ( X | Θ ) , p d ( X | Θ )) is γp md -local, where p md = Mix ( p m ( X | Θ ) , p d ( X | Θ ( d ) ) ) p ( Θ ) .

Proof. Similar to WS, the update is given by:

where in this last equality, X ∼ p md ( X ) , and we are performing a single-sample gradient approximation. It is worth noting that unlike in the Wake-Sleep algorithm, here γ t is not a constant throughout time. Instead, γ t alternates between 0 and 1 with 'phase duration' K , i.e. γ t +1 = 1 -γ t if mod ( t, K ) = 0 , and γ t +1 = γ t otherwise. The IL update is the score function of p md ( X | 1 -γ, Θ ) , which has identical dependencies to the score function of p md ( X | γ, Θ ) (only a change from γ → 1 -γ has occurred). Therefore, if p ( Θ ) = ∏ k p ( Θ k ) , this parameter update is γp md -local by Property 2.7.

E.7 Contrastive Divergence for Boltzmann machines (CD)

While the GEM learning update is provably p m -local, it is also predicated on the assumption that the parameter marginal distribution factorizes as p ( Θ ) = ∏ k p ( Θ k ) , which as we note in Appendix D can be difficult to ensure for even simple undirected graphical models. An extension of the GEM algorithm, we can show that the CD algorithm is γp md -local (as opposed to just p m -local) under less restrictive assumptions. The cost of this is that CD learning usually requires costly MCMC sampling from both the posterior distribution p m ( X h | X o , Θ ) and the full joint distribution p m ( X h , X o | Θ ) .

Theorem E.7. The CD update given by A CD ( p m ( X | Θ ) , p d ( X )) is γp md -local, where p md = Mix ( p m ( X | Θ ) , p d ( X )) p ( Θ ) .

As mentioned above, for GEM the most natural choice for p d ( X h | X ) is given by p m ( X h | X , Θ 0 ) . As we demonstrated, parameter updates calculated according to this rule will be γp md -local, but there are two important caveats. First, for GEM to produce biologically plausible updates, we still need a biological system that can sample from p m ( X h | X , Θ 0 ) . Second, it is important to remember that we are only guaranteed that the score function is guaranteed to be p -local if the marginal parameter probability distribution factorizes as p ( Θ ) = ∏ k p ( Θ k ) . For a DAG, it may be difficult to satisfy the first condition without approximation (given by the Wake-Sleep algorithm, for instance), whereas for an UG, it may be difficult to satisfy the second condition, as we saw in Section D. To make our update γp md -local for an undirected graphical model like the Boltzmann machine, we will require an extra step that we outline here to use the energy function Property 2.6 rather than the score function Property 2.7.

Having committed to working with an undirected graphical model, instead of sticking to the original GEM update, here we break apart the probability distribution as:

̸

where X k is sampled from p md . Now, by Property 2.6 and Property 2.8, this update is γp md -local. This is the Boltzmann machine learning algorithm [26], where the clamped and unclamped phases are alternated between stochastically [46]; for our previous linear Boltzmann example, in which X = r and Θ = W , the derivative of the energy function (Eq. D.3) with respect to a parameter W ij is r i r j , demonstrating that updates correspond to two phases of updates: one in which r o is clamped to a data distribution given by p d ( r o ) for some r o ⊆ r and Hebbian updates are positive, contrasted with an unclamped phase in which updates are negative. Note that for this model approximate sampling from a posterior distribution p m ( r = o | r o , W ) p d ( r 0 ) is no more difficult than sampling from the joint distribution: one simply holds r o fixed to an environmental data sample and performs Langevin sampling on all other variables.

E.8 Equilibrium Propagation (EP)

Though the derivation and setup of the equilibrium propagation algorithm [27] is very different from Contrastive Divergence, the functional form of the derived update is very similar. While equilibrium propagation typically operates on deterministic networks, here we will provide our derivation for the stochastic version with an energy function defining a joint distribution over Θ and X (as in Section D), which is somewhat more straightforward to fit into the p -locality framework.

Suppose that we have a probabilistic energy-based model whose energy function is given by:

where ϵ ∈ { 0 , β } , where β ≪ 1 , where L ( Z ) is the loss function that the parameter updates are optimizing. This can be thought of as a 'soft-clamped' system, in which nonzero ϵ pushes the system towards slightly better performance. Intuitively, the EP parameter update attempts to change network dynamics so that the unclamped system is nudged towards the slightly better performing soft-clamped system. Then we have the following theorem:

Theorem E.8. The EP update given by A EP ( p ( X | Θ , γ )) is γp md -local, where p md = Mix ( p ( X | Θ , ϵ = β ) , p ( X | Θ , ϵ = 0)) p ( Θ ) .

The parameter update for equilibrium propagation is given by:

where for the final equality we are using a sampling-based approximation in which we are sampling from X k ∼ p ( X | Θ , ϵ k ) , and ϵ k /β ∼ Bernoulli (0 . 5) . This is almost identical to the Contrastive Divergence update, except that rather than clamping neural activities to a target output, they are slightly biased towards better performance. Because this is the combination of the derivative of the energy function with a mixture variable γ = ϵ k /β , by Properties 2.6 and 2.8, this update is γp md -local where p md = Mix ( p ( X | Θ , ϵ = β ) , p ( X | Θ , ϵ = 0)) p ( Θ ) .

E.9 Winner-take-all STDP

While Contrastive Divergence uses MCMC sampling to approximate the GEM update, Nessler et al. [48] use a particular generative model for which the posterior can be analytically calculated and resembles a simple winner-take-all neural circuit. Then, the authors derive their STDP parameter update as an approximation to the GEM algorithm. Because of this, one might imagine that the derived STDP update may, like the GEM algorithm, be p m -local. We will see below that this is the case.

First, we define the generative model p m used in the paper:

̸

where Z is the normalizing constant and the r and s vectors contain binary random variables. Furthermore, in the network only one neuron r is assumed to fire at any given time ( r i = 0 ⇔ r k = 0 ∀ k = i ). The inference distribution, conditioned on a stimulus s can be calculated as follows:

̸

This probability distribution can be interpreted as a kind of winner-take-all computation, dominated by the neuron with the greatest input current [48]. Samples from this distribution are used to compute the weight updates:

where c is a positive constant. We now prove the following theorem assessing the p -locality of this distribution:

Theorem E.9. The STDP update given by A STDP ( p ( X | Θ )) is γp m -local.

Proof. To see this, we first note that the gradient of the energy function E ( r , s , W ) with respect to the parameters is p m -local by Property 2.6. Therefore, any variables contained within this will also be permissible under p m -locality.

For W ij , we have:

so that we know r i and s j are permissible for ∆ W ij under p m -locality; further, the value of a parameter itself, W ij , is always allowed under p m -locality. These are the only variables on which ∆ W ij depends, so this update is p m -local.

For W i 0 , we have:

so that we know r i is permissible for ∆ W i 0 . By the same reasoning, this update is also p m -local. Since all updates are therefore p m -local, we may conclude that the full algorithm is p m -local. However, this proof does not have the same level of generality as for the previous algorithms, because the algorithm is only defined for a single winner-take-all network model.

E.10 Backpropagation

Theorem E.10. If p ( Θ ) = ∏ k p ( Θ k ) and p ( X | Θ ) is defined by Eq. 3 (with X = r and Θ = W ), the BP update for W ( l ) ij with a loss L ( r ) , given by A BP ( p ( r | W ) , L ( r )) is e ( l ) i p -local, where e ( l ) i = d L d¯ r i . Similarly, the updates for feedback alignment, weight mirror, and Burstprop are ˆ e ( l ) i p -local, where ˆ e ( l ) i is given by their respective gradient approximations.

Proof. As a first step, for clarity purposes we will demonstrate that backpropagation [28], is not p -local with respect to the simple feedforward neural network architecture we outlined above; we will subsequently demonstrate that it and its approximations do satisfy a particular notion of S p -locality. For a scalar loss function L ( r L ) and a single parameter W ( l ) ij , the backpropagation gradient is given by the negative gradient of the loss with respect to the parameter of choice, using the reparameterization trick [25, 24] to take for a single sample of { r ( l ) } l =0: L the mean (noiseless) mapping from r ( l -1) → r ( l ) to be ¯ r ( l ) = h ( W ( l ) r ( l -1) ) , so that by Eq. 5, we have r ( l ) = ¯ r ( l ) + σ η ( l ) . By the chain rule, gradient descent gives:

Based on our analysis in Section 2.4, this update function is clearly not p -local, because the update depends on firing rates r ( k ) for k > l . However, while backpropagation is not in general p -local, any algorithm can be S p -local: for example, if we take S = Z , then by Definition 2.3, any parameter update can contain any variable in the graphical model p ( Z ) . Taking S = Z is inherently vacuous: S p -locality is only conceptually useful if we can cleanly reduce the number of variables included in S for a broad set of biologically relevant neural architectures. Fortunately, for backpropagation

operating on a feedforward neural network governed by Equation 5, we do not need to include every variable in the network. By Equation E.36, we see for our example feedforward neural network that:

where d L d¯ r ( l ) i = d L ( r ( L ) ) d¯ r ( L ) ( ∏ L k = l +2 d¯ r ( k ) ( r ( k -1) ) d¯ r ( k -1) ) d¯ r ( l +1) ( r ( l ) ) d¯ r ( l ) i is the derivative of the global loss function with respect to the individual mean neuron activation ¯ r ( l ) i . Interestingly, d¯ r ( l ) i d W ( l ) ij -the derivative of the mean parameter for neuron r i -is a function of only the parents of r ( l ) , which are therefore the coparents of W ( l ) ij . To verify that this particular component of the weight update is p -local, we can compare its dependencies to the score function, which is in this case p -local by Property 2.7. As noted in Section 2.4, the score function is given by:

where V ( l ) i = W ( l ) i : r ( l -1) . Because the score function is p -local, any variables that it depends on are permissible for p -local updates. The score function depends on r ( l ) i and r ( l -1) , whereas d¯ r ( l ) i d W ( l ) ij depends only on r ( l -1) . It follows that this function is also p -local.

As we already discussed, d L d¯ r ( l ) i is not p -local because it depends on neurons downstream of r ( l ) i . However, if we define an auxiliary random variable e ( l ) i = d L d¯ r ( l ) i , we see that because it multiplies e ( l ) i with a p -local function, W ( l ) ij is e ( l ) i p -local.

Importantly, this does not mean that backpropagation is biologically plausible: this notion of locality provides no clues as to how e ( l ) i could be calculated or approximated in the brain, and an explicit calculation of gradients could not be possible due to the weight transport problem [20]. There are many recent models that account for how e ( l ) i could be approximated by an approximate credit assignment signal ˆ e ( l ) i involving either random feedback synapses that project errors backwards through the network (feedback alignment [29]) or feedback synapses that dynamically adjust through local synaptic mechanisms so that ˆ e ( l ) i provides an unbiased approximation (e.g. weight mirror or Kolen-Pollack alignment [30], and BurstProp [31]). Each of these algorithms decomposes into a nonlocal feedback term ˆ e ( l ) i and a p -local term in exactly the same way, and are consequently ˆ e ( l ) i p -local.

E.11 Real Time Recurrent Learning (RTRL)

Consider an autonomous recurrent neural network whose directed acyclic graphical model is provided by the following equations (we will ignore stimulus-dependence for notational simplicity):

where p ( r (0)) corresponds to some initial distribution of activity states. This probability distribution of firing rates corresponds to the following neural sampling dynamics:

where η ∼ N (0 , 1) . For this model, we have the following theorem:

Theorem E.11. If p ( Θ ) = ∏ k p ( Θ k ) and p ( X | Θ ) is defined by Eq. E.39 (with X = r and Θ = W ), the RTRL update for W ij with a loss L ( r ( T )) , given by A RTRL ( p ( r | W ) , L ( r )) is eJ p -local, where e = ∂L ( r ( T ) ∂ ¯ r ( T ) , and J = { J ( t ) = ∂ ¯ r ( t, r ) ∂ ¯ r ( t -1 , r ) } .

Proof. The directed graphical model corresponding to these dynamics is depicted in Figure 1b: as with backpropagation, we will use the score function for our graphical model to identify permissible variables. For a single synapse, the score function is given by:

̸

where V i ( t ) = W i : r ( t -1) . Thus p -local parameter updates for W ij may include W i : , r i ( t ) and { r k ( t -1) : W ik = 0 } ∀ t . We will now compare these allowed variables to the RTRL update. As with backpropagation, we take ¯ r ( t, r ) = h ( Wr ( t -1)) , so that r ( t ) = ¯ r ( t, r ) + σ η . The RTRL update minimizes a loss L ( r ( T )) via the chain rule [49, 21]:

̸

this second equation provides a recursive update equation which can be stored online as a trial progresses. The g ( r ( t -1)) term is p -local, because it appears in Eq. E.43. However, ∂ ¯ r ( t, r ) ∂ ¯ r ( t -1 , r ) , an N × N Jacobian matrix, is not p -local, since it depends on all neurons in the network r ( t -1) as well as all parameters W -neurons that do not directly synapse onto neuron r i and weights W kl for k = i are excluded from p -local updates by Property 2.1 according to the DAG defined by Eq. E.39. Furthermore, as we have seen with backpropagation, in general the credit assignment signal ∂ L ( r ( T )) ∂ ¯ r ( T ) is not p -local. Therefore, to characterize the S p -locality of RTRL, we will have to proceed similarly to backpropagation, and define auxiliary variables to include in the set S .

As with backpropagation, we define the auxiliary random variable e = ∂ L ( r ( T )) ∂ ¯ r ( T ) . Because we have found the Jacobians to also violate p -locality, we will also define the set of auxiliary variables J = { J ( t ) = ∂ ¯ r ( t, r ) ∂ ¯ r ( t -1 , r ) } . With these auxiliary variables, we can see that ∂ ¯ r ( t, r ) ∂ W ij is J p -local ∀ t , and consequently, the RTRL update is eJ p -local.

This is, of course, not biologically plausible in any way. The set J allows the parameters to have access to the state of the entire network, at all time points, even from neurons that do not have any direct connections to the neuron whose synapse is being updated. Further, the entire error vector e is required to compute the update. This is even less plausible than backpropagation, which only required access to e i . However, the RTRL update is an important baseline for analyzing the locality properties of other learning algorithms that are constructed as approximations of it, namely e-prop and RFLO.

E.12 e-prop

Theorem E.12. If p ( Θ ) = ∏ k p ( Θ k ) and p ( X | Θ ) is defined by Eq. E.39 (with X = r and Θ = W ), the e-prop update for W ij with a loss L ( r ( T )) , given by A ep ( p ( r | W ) , L ( r )) is e i p -local, where e i = ∂L ( r ( T ) ∂ ¯ r i ( T ) .

Proof. For e-prop [37], we will also consider networks constructed according to Eq. E.39. The update is almost identical to the RTRL update, but several terms will be discarded, allowing the update to be e i p -local, as opposed to the eJ p -local update given by RTRL. The update is as follows:

This update combines the neuron-specific credit assignment signal e i with a local 'eligibility trace' ∂ ˜ r i ( t, r ) ∂ W ij which performs approximate credit assignment by filtering and summing coactivity between neuron i and neuron j across timesteps. It is worth noting that the particular functional form of this eligibility trace is determined by our simplified RNN dynamics (Eq. E.39), which causes coactivity from previous timesteps to decay exponentially in proportion to the magnitude of the autapse W ii -alternative neural network dynamics using continuous-time dynamics, or adaptive neural firing thresholds may alter the functional form of the eligibility trace [37], but do not fundamentally alter the p -locality properties of the update. Now, we only need to show that the eligibility trace is p -local.

As with RTRL, we can observe that h ′ ( V i ( t )) and r j ( t -1) both appear in the score function for our RNN (Eq. E.43) for all timesteps, as does W ii ⊂ W i : . Because the score function is p -local, we know that these variables are all allowed under p -locality. The eligibility trace only depends on these terms, from both the current time step and, recursively, from previous timesteps. Therefore, the eligibility trace is p -local. The e-prop update is a multiplication between e i and the eligibility trace, so by Def. 2.3 the update is e i p -local.

E.13 Random feedback local online learning (RFLO)

The RFLO update [36] is nearly identical to the e-prop update, except we replace e i with an approximate credit assignment signal ˆ e i (which replaces symmetric feedback weights with random connections, similar to Feedback Alignment).

The update is given by:

Following exactly the same reasoning as with e-prop, we may show that this update is ˆ e i p -local.

Theorem E.13. If p ( Θ ) = ∏ k p ( Θ k ) and p ( X | Θ ) is defined by Eq. E.39 (with X = r and Θ = W ), the RFLO update for W ij with a loss L ( r ( T )) , given by A RFLO ( p ( r | W ) , L ( r )) is ˆ e i p -local, where e i = ∂L ( r ( T ) ∂ ¯ r i ( T ) .

E.14 Feedback-based Online Local Learning Of Weights (FOLLOW)

The FOLLOW algorithm [50] is defined in terms of a particular continuous-time LIF circuit with postsynaptic potential kernels. For simplicity, we will focus our analysis on a linear version of the same circuit, disregarding the dynamic postsynaptic potentials and input stimuli. Disregarding these features does not affect the p -locality properties of the FOLLOW algorithm, but it would certainly degrade its performance on tasks.

The network dynamics are given by:

where W fb is an N × N o random feedback weight matrix, k is a positive constant, and e ( t ) is an N o -dimensional error feedback vector delivered at every timestep, with N o the number of output dimensions. Because we are in continuous time, we will assume that η ∼ N (0 , ∆ t ) .

Similar to RTRL, we can write the probability distribution for the network as:

where p ( r (0)) is some initial distribution of firing rates. Further, we can assume that the distribution of errors at timestep t +∆ t has any arbitrary distribution p ( e ( t +∆ t ) | r ( t )) .

The update for weight W ij is given by:

Therefore, only the postsynaptic error current and presynaptic input are necessary to update the weights for a given synapse in this type of network. Below, we will show that this update is p -local. Theorem E.14. If p ( Θ ) = ∏ k p ( Θ k ) and p ( X | Θ ) is defined by Eq. E.53 (with X = { r , e } and Θ = W ), the FOLLOW update for W ij , given by A FW ( p ( r , e | W )) is p -local.

Proof. To see that this is true, we need only show that the variables included in ∆ W ij are subsets of the variables included in the score function ∂ log p ( r , e | W ) ∂ W ij . These variables are permissible for p -local updates by Property 2.7. The score function is given by:

̸

Therefore, for weight ∆ W ij , the permissible variables include: r i ( t ) ∀ t , any r k ( t ) such that W ik = 0 ( ∀ t ), any e k ( t ) such that W fb ik = 0 ( ∀ t ), and the parameters W i : and W fb i : . The parameter update requires only r j ( t ) and W fb i : e ( t ) , which is a subset of these permissible variables. Therefore, the update is p -local.

̸

F p -locality does not guarantee biological plausibility

It is very important to clarify the exact relationship between p -locality and biological plausibility. Except for some network-wide variables that a theoretician may decide to allow through a particular choice of S p -locality, we have generally shown that p -locality is overly permissive , in that a particular choice of p may allow parameter updates to include variables that an individual synapse may not have access to. Furthermore, p -locality does not restrict the network architecture defined by p to be biologically plausible. The best way to interpret p -locality is as follows: if p ( X , Θ ) defines a biologically plausible architecture and an algorithm A is p ( X , Θ ) -local, then the parameter update provided by A ( p ( X , Θ )) will be biologically plausible. There are many network architecture and parameter update combinations that may be biologically plausible without being proven p -local (e.g. explicit approximations to backpropagation [29, 30, 31]), as there are many combinations that are p -local without being biologically plausible. Below, we will show two important instances in which p -locality does not properly diagnose a combination of network architecture and parameter update as biologically implausible.

F.1 Locality and architectural plausibility

The first example is pervasive in neural network models of the brain: networks frequently violate Dale's law, which states that neurons in a neural network are (for the most part [51]) either excitatory

̸

(outgoing weights are positive) or inhibitory (outgoing weights are negative), but not both. In fact, in the simple network example we have provided (Section 2.4), neural firing rates are not constrained to be strictly positive, and outgoing synaptic weights are not sign-constrained. For this biologically implausible architecture, p -locality defines which variables are allowed to be included in individual parameter updates in a way that is sensible (allowing only variables involving the postsynaptic firing rate and the firing rates of all pre-synaptic neurons), but it says nothing about the aforementioned implausibilities of the network architecture. Similarly, the linear Boltzmann machine example provided in Appendix D does not constrain firing rates to be positive, and requires symmetric weights ( W ij = W ji ), which could not satisfy Dale's law while allowing connections between an inhibitory neuron i and an excitatory neuron j ( W ij > 0 while W ji < 0 ⇒ W ij = W ji ).

These examples illustrate an important fact: p -locality focuses on the plausibility of updates given an architecture that has been predetermined to be acceptable. However, it is worth noting that if we were to impose these additional constraints for the proposed networks, the accepted variables determined by p -locality would not change.

F.2 Parameterizing probabilities with neural networks

Another important caveat when working with p -locality is that the random variables Z have to correspond to the relevant biophysical quantities of interest, e.g. neural firing rates X and synaptic weights Θ . If this is not the case, then p -locality can easily defy standard notions of biological plausibility. For instance, if we define a probability distribution in terms of a 3-layer neural network:

Then the score function of this distribution is given by:

This equation depends on X , which is the output of the network, even though Θ parameterizes ¯ r (0) . Therefore, if the random variables had been defined in as in Section 2.4, then this update would not be p -local. However, because it is the derivative of the score function, for any independent marginal p ( Θ ) , it is p -local for this choice of random variables. Therefore, it is important when working with an algorithm such as Wake-Sleep or REINFORCE, that one chooses a conditional probability distribution p ( X | Θ ) that captures biologically plausible dependencies. When this is not done, as in [52, 53], the resulting updates have no correspondence to synaptic plasticity rules.

Note that this fact does not undermine the utility of p -locality as a concept. Our proofs for algorithms in Appendix E apply for any p ( X | Θ ) , as long as p ( Θ ) factorizes to ∏ i p ( Θ i ) . Therefore, algorithms that have universal p -local properties will respect the variable dependencies implied by p ( X | Θ ) whether this distribution is plausible or not, which means that the algorithms will respect variable dependencies for all plausible network architectures.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nMany biologically plausible learning algorithms have been developed in the recent years but much less effort have been deployed to compare them and understand how they could yield experimentally testable hypothesis. The present work introduces a framework that allows comparing existing algorithms. In particular, the authors introduce the concept of Sp-locality which highlights the quantities needed to be available to synaptic plasticity, the p part focusing on quantities made available by neural computations and the S one on additional signals learning signals. Different groups of plasticity rules arise from the proposed analysis, helping formulating empirically falsifiable hypotheses.\n\n\n**Score increased from 6 to 7 after the rebuttal**.\n\nSTRENGTHS:\nThe paper is technically sound, introducing formal definitions of locality and thoroughly examining their implications. It is well written, with examples and illustrative figures facilitating reader comprehension of the newly introduced concepts. The analysis conducted allows a straightforward comparison of various learning rules. Furthermore, the authors transparently outline the framework's limitations.\n\nThose points make the paper valuable to the NeurIPS community, particularly due to the lack of prior work on the question it addresses.\n\nWEAKNESSES:\nSome of the properties deriving from Sp-locality do not seem particularly aligned with what one could picture of biological plausibility. As this may result from a misunderstanding from my side, I address those points in the question section. I am happy to improve my score once those questions are addressed.\n\nQUESTIONS:\n**Note:** all the following questions are based on my current (incomplete) understanding of the paper.\n\nIn a feedforward neural network, a plasticity rule for $W^l_{ij}$ (synaptic weights between layer $l-1$ and $l$) that depend on $r^{l-1}$ (activity of the **entire** $l-1$ layer) and of $r^{l}_j$ is consider $p$-local. Is this compatible with any notion of biological plausibility? This point makes me wonder whether the notion of $p$-locality is fully aligned with locality in physical space.\n\nWhat are the main differences of the proposed approach with an alternative one focusing on the computational graph of neural computation? Such an approach would, for example, not suffer from the FF issue described above. It would start from the neural computations performed and end up with the learning rules. Of course, one downside is that it now depends on the details of neural computation.\n\nHow would the Sp-locality framework extend to algorithms attacking the temporal credit assignment problem, e.g. e-prop of Bellec et al. 2020 or FOLLOW of Gilra and Gerstner 2017?\n\nFinally, which conclusions the framework enables establishing that a more vanilla approach that just analyze the learning rules “manually” does not achieve?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-06-19"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper describes a systematic and mathematically formal method for categorizing learning algorithms into different types and degres of locality. It also applies the method to certain such algorithms and determines their type of locality.\n\nSTRENGTHS:\nThe authors chose to work on a topic that is indeed very important, and in a field has been very much in need of systematic evaluation. The claims of biological plausibility and locality in the recent literature have been increasing and widening, so a method to validate such claims has been necessary.\n\nThe fact that the paper does not merely provide a verbal definition of types of plasticity but rather it attempts to make it mathematical is very interesting.\n\nThe framework is mostly reasonable, judging by its results when applied to the categorization of specific learning rules.\n\nWEAKNESSES:\nThe weakest aspect of the framework presented here is that it falls victim to one of the issues that it intends to resolve.\n\nMore specifically, algorithms such as equilibrium propagation or predictive coding require that the network reach a global equilibrium through multiple forward and backwkard propagations before a training example completes its updating action on the network parameters. This is arguably less local than even standard backpropagation, as it requires not only that global signals propagate through the network once, but indeed multiple times. This has been recognized in recent works (https://iopscience.iop.org/article/10.1088/2634-4386/aca710, https://openreview.net/forum?id=8gd4M-_Rj1) and this recognition has even led to a formal proof that the time-complexity of these algorithms is lower-bounded by that of backpropagation (https://arxiv.org/abs/2304.02658). Therefore, this type of locality is clearly not the same as that of, for example, Contrastive Divergence for Boltzmann machines or Deep Belief Networks which do not presuppose such a global and multilayer equilibrium. In turn, even in the cases where such an equilibrium requires iteration only within individual layers and where greedy layer-wise training is possible, e.g. in Boltzmann machines and contrastive Hebbian rules, the final weight update cannot be said to be equaly local to that of simpler Hebbian rules that do not require such iterative processes or feedback, and complete the update truly locally in a single timestep. The distinction of these categories of locality is crucial. Disregarding these in a paper that claims to resolve such confusions risks reinforcing and perpetuating them instead.\n\nThe above issue of the paper might have been caused by the fact that many of the simplest and most local learning rules are not normative but rather are based on heuristics, whereas the presented framework is concerned with normative rules. However, there do exist such rules that are normative, eg the type of STDP in Nessler et al., NIPS 2009/Plos Comp. Biol. 2013; the work of Pehlevan & Chklovskii, NIPS 2015; and SoftHebb in Moraitis et al., Neuromorph. Comput. Eng. 2022. In fact, what appears to be the current state of the art in biological plausibility of deep learning and in performance of bio-plausible deep learning (Journé et al., ICLR 2023) is one of the above normative rules, namely SoftHebb.\n\nTherefore, I recommend that the authors adjust their framework to account for the substantial differences in locality between the three categories of algorithms that are exemplified by (a) Equilibrium Propagation, (b) Contrastive Divergence/Boltzmann Machine, and (c) SoftHebb or STDP. This should be accompanied by making sure that examples of all these categories are included in Table 1.\n\nAs a minor note, the paper's Table 1 mentions Boltzmann Machines as an algorithm, but BMs are models. I believe Contrastive Divergence is meant by the authors, and if so it should be corrected.\n\nMore generally, the fact that the current formalization suffers from similar issues as other less formal claims in the literature, reveals that perhaps the mathematics in this case are superfluous, since its conclusions depend mainly on the assumptions, which in turn depend on the categories that the authors intend to achieve. In other words, I suspect that the main benefit of a systematic categorization could be achieved without mathematical terms. Could the authors please comment on this? What do they regard as added value from the use of mathematics instead of mere natural language and logical arguments in this case?\n\nRelated to this, the paper would be strengthened significantly if both the framework and the resulting categories of locality could be described in a dedicated paragraph in natural language, with an intuitive explanation.\n\nQUESTIONS:\n\"We subsequently use this framework to distill testable predictions from various classes of biologically plausible synaptic plasticity models that are robust to arbitrary choices about neural network architecture\"\nThis is a quote from the abstract. Could the authors please elaborate on this? What exactly are the testable predictions? Perhaps this refers to the last paragraph of the paper, but that paragraph is not entirely clear to me, and it also seems like it does not make any specific prediction.\n\nIf the authors could address these issues both in the revised manuscript and in their rebuttal here, I am open to re-evaluating the submission.\n\nLIMITATIONS:\nThere is no discussion of the paper's limitations in the paper.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a general definition of locality of updates in neural networks, which is an important requirement of biological plausibility. Variants of locality requirements allow to describe classes of networks and algorithms. Several important algorithms are analyzed through this lens.\n\nSTRENGTHS:\nDefinition of locality (and variants) is novel (to my knowledge), considerations are mathematically sound.\n\nThe paper outlines several patterns of locality in existing algorithms, which deserves interest.\n\nWEAKNESSES:\nThe need for the general definition of locality is not substantiated sufficiently. Was that need already expressed in the literature? Were there other attempts? Is there an example where such definition would resolve a controversy?\nWhenever a model is formally defined, its biological plausibility (which of course goes beyond locality) is up for debate. However within the model the information available for parameter updates must be clear from the definition (or I can't think how it can not be). Therefore I am not convinced that suggested framework will help \"guide claims of biological plausibility\", as the paper says.\n\nIt is not quite convincing that introducing probabilities is worth the trouble. After all, the answer to locality question is always binary: yes or no. It seems that definition through graphs (directed or undirected) would be sufficient in overwhelming majority of cases, and whenever the probabilities are already defined, the graphical model can be referred to.\n\nQUESTIONS:\nThe authors claim there is lack of clarity about locality for some models. Can you give an example?\n\nDefinition 2.1: line 87, what are you trying to say by underscoring the word 'direct'? what would be indirect?\n\nFig. 1b Node r_j for t_2 is not a child or coparent of W_{ij} but not excluded.\n\nColoring in figures can be guessed, but better be explained in words.\n\nEq.4 How would the definition 2.2 work when $h$ is not differentiable, e.g. ReLU?\n\nline 239 A(p(X, \\Theta)) = f(Z) - it looks like update function is somehow fully determined by the probability distribution? Please define the meaning of A(p(...)) here and in theorems 3.1, 3.2.\n\nLIMITATIONS:\nNo problem nere", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose \"Sp-locality\" as a formal definition of locality in models of synaptic plasticity, as well as two intermediate definitions, S-locality and p-locality. S-locality explicitly enumerates the set of variables which directly participate in a synaptic update for each synapse. p-locality replaces the explicit set S with the Fisher information of that variable (conditioned on all other variables to eliminate indirect dependencies) with respect to the synapse. Finally, Sp-locality expands this by measuring influence as defined by p-locality, additionally allowing a dependence of the synaptic update on an explicit set of variables as with S-locality (although eliminating the need for an exhaustive enumeration). Finally, the authors apply these definitions to a number of well-known models of synaptic plasticity, delineating four major classes of models and suggesting experimental predictions.\n\nSTRENGTHS:\nI am unaware of similar work attempting to organize plasticity rules according to their locality, although I am interested to see what other taxonomies have been suggested. The paper is exceptionally well written, building up the concept of Sp-locality from simpler intuitive concepts, providing examples and intuitions, and expanding on these in the Appendices. The authors classify a wide range of models using their framework and show useful mappings between the graphical model and their definitions of locality. Overall, this paper is a useful initial effort to formally categorize \"local\" plasticity rules, and is a valuable conceptual contribution and starting point for follow-up work in this direction.\n\nWEAKNESSES:\n- Although the authors do a thorough job of applying their framework to a diversity of plasticity models, I am curious about further contextualization of the work within meta-topic of \"taxonomy of plasticity rules\"\n- As the authors point out, p-locality does not imply biological plausibility (nor vice versa). Sp-locality is an even weaker constraint, as it allows for not only p-locality but also dependence on an explicit subset S of network variables. These definitions punt the problem of \"biologically plausible plasticity\" to the problem of defining \"biologically plausible architecture,\" which is again often defined intuitively and ad-hoc. As such, the practical utility of this framework is unclear to me. \n- It is unclear to me how this framework adds power to experimental predictions beyond the ones made by the individual plasticity rules and their ad-hoc appeals to biological plausibility. I would be interested to see more discussion about this with explicit examples of predictions which can be made with this framework and followup experiments that can be performed (even conceptually, disregarding their practical feasibility), or even a retrospective \"post-diction\" of a known experimental phenomenon which could have been predicted by this framework.\n\nQUESTIONS:\n- Is there other work attempting a conceptually similar grouping of plasticity rules based on the information used by synapses?\n- Can these definitions account for temporal locality or lack thereof? e.g. how would backpropagation through time or its biologically plausible approximations fit into this framework? \n- Is there a mapping between p-locality and S-locality? i.e. given a p-local rule (for a particular architecture), is there always a way of defining S-locality, and vice versa? (Is this even a useful mapping to make?)\n- Are there plasticity models that do not fit into this framework (excepting the trivial cases in which S=Z)?\n\nLIMITATIONS:\nThe authors carefully analyzed the limitations of their definitions, although as far as I can tell these limitations severely handicap the practical utility of the proposed framework. Due to the basic science nature of this work, potential for direct societal impact is minimal.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-25"}