Dis-inhibitory neuronal circuits can control the sign of synaptic plasticity
Authors: Julian Rossbroich
Friedemann Zenke
Venue: NeurIPS 2023 poster (2023)
Keywords: Credit assignment
hebbian plasticity
inhibitory microcircuits
bio-plausible learning

Abstract: How neuronal circuits achieve credit assignment remains a central unsolved question in systems neuroscience. Various studies have suggested plausible solutions for back-propagating error signals through multi-layer networks. These purely functionally motivated models assume distinct neuronal compartments to represent local error signals that determine the sign of synaptic plasticity. However, this explicit error modulation is inconsistent with phenomenological plasticity models in which the sign depends primarily on postsynaptic activity. Here we show how a plausible microcircuit model and Hebbian learning rule derived within an adaptive control theory framework can resolve this discrepancy. Assuming errors are encoded in top-down dis-inhibitory synaptic afferents, we show that error-modulated learning emerges naturally at the circuit level when recurrent inhibition explicitly influences Hebbian plasticity. The same learning rule accounts for experimentally observed plasticity in the absence of inhibition and performs comparably to back-propagation of error (BP) on several non-linearly separable benchmarks. Our findings bridge the gap between functional and experimentally observed plasticity rules and make concrete predictions on inhibitory modulation of excitatory plasticity.

Dis-inhibitory neuronal circuits can control the sign of synaptic plasticity
Julian Rossbroich"
Friedemann Zenkel
(firstname lastname)@@fii ch
Friedrich Miescher Institute for Biomedical Research, Basel, Switzerland 2 Faculty of Science, University of Basel, Basel, Switzerland
Abstract
How neuronal circuits achieve credit assignment remains a central unsolved ques- tion in systems neuroscience. Various studies have suggested plausible solutions for pack-propagating error signals through multi-layer networks. These purely functionally motivated models assume distinct neuronal compartments to repre- sent local error signals that determine the sign of synaptic plasticity. However, this explicit error modulation inconsistent with phenomenological plasticity models in which the sign depends primarily postsynaptic activity. Here we show how a plausible microcircuit model and Hebbian learning rule derived within adaptive control theory framework can resolve this discrepancy. Assuming errors are encoded in top-down dis-inhibitory synaptic afferents, we show that euror-modulated learning emerges naturally the circuit level when recurrent inhibition explicitly influences Hebbian plasticity. The same learning rule accounts for experimentally observed plasticity in the absence of inhibition and performs comparably to back propaaaapattnnn of error (BP) on several non-linedd con-linearly separable benchmarks. Our findings bridge the gap between functional and experimentally observed plasticity rules and make concrete predictions on inhibitory modulation of excitatory plasticity.
1 Introduction
How do neurons far away from the sensory periphery and motor output system update their con- nections contribute network computation meaningfully? This question, formally known as the "credit assignment problem," is one of the outstanding questions in systems neuroscience. Clas- sic learning theories assume synaptic plasticity in the brain is mainly Hebbian, i.c., changes in synapse's is efficacy depend merely on the pre- and postsynaptic activity of the neurons it connects [i]. A plethora of experiments in different brain areas support the notion of Hebbian plasticity [2, 3] and diverse phenomenological plasticity models quantitatively capture observed synaptic plasticity dynamics [4-8]. Theories of reinforcement learning suggested further extensions Hebbian learning to three-factor rules that account for reward modulation, whereby global modulatory factor, e.g., dopamine, explicitly influences the sign plasticity [9, 10]. Again, there is ample experimental evidence for the role of neuromodulators in gating plasticity in various brain recent work argues that global neuromodulation is insufficient to learn complex function mappings in large networks [13, 14] and that more fine- fine-graine grained control over the sign of plasticity is required, in the BP algorithm used in deep learning [15-18]. This realization motivated several modeling studies on how biological networks could approximate BP [19-25). One central assumption in virtually all of these models is that the sign of plasticity explicitly modulated by a neuron-specific local error signal akin the gradient-based update used in BP [26]. However, learning rules with explicit error
[11, 12]. Yet,
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

a) Experimentally observed b)
Normative theories of gradient-baser learning
Backpropagation
Dendritic error coding
Hebbian plasticity
1op-down
OOO
forward
1000
0 Peedback
ac Öpost,
6/ x
6,
Awiy pre, x (post,,
Awyy pre, x g(post,) cs
Equilibrium propagation
Burst deirndde plasticity
LTP
Aniia
Amig
1/2
X ou
LTD
6: 0c (s(ti)
uc((2)
post,
post,
Figure 1: Explicit enror modulation of the sign of plasticity is inconsistent with phenomenological plasticity models. (a) In neuronal circuits, top-down feedback connections target excitatory neurons, well as inhibitory and dis-inhibitory circuits that have been implicated in gating of plasticity. In phenomenological plasticity models, the sign of plasticity is typically determined by postsynaptic quantities such the membrane voltage [7], firing rate [6], or calcium concentration [8] without explicit error modulation. (b) In normative models, the sign of plasticity is typically subject to modulation with little experimental evidence. Explicit error modulation makes specific predictions of the shape of the learning rule, (Supplementary Fig. S1; see Appendix A), odds with experimentally observed plasticity (see Panel (a)). (b) Theories of bio-plausible gradient- based learning typically focus on approximating direct error- mror-modulation as in BP, but differ in how errors are computed and relayed. Existing models suggest separate temporal phases, c.g. equilibrium propagation (EP) [29], putative compartments to represent error signals locally. e.g., dendritic error coding [22, 30], or burst-multipeexing [23, 31]. Still, there little evidence for an explicit error signal that alters the sign of plasticity [26] and is consistent with phenomenological plasticity models.
hypothetical, explicit
modulation are inconsistent with phenomenological models of Hebbian plasticity (Fig. I), raising the question of how theories of bio-plausible credit assignment tie into the phenomenology of Hebbian learning.
Here, we address this question using normative control theory approach. We set out from plausible dis-inhibitory circuit motif ubiquitously found in the brain [27, 28] and derive a Hebbian learning rule with an explicit inhibitory current dependence. We demonstrate that this learning rule accounts for key experimental observations and allows for control over the sign of plasticity through top-down synaptic input specific interneurons. Our work suggests that error signals could naturally be encoded in top-down inputs to inhibitory neurons and bridges the gap between normative theories of gradient-based learning and phenomenological models of Hebbian plasticity. Our main contributions are:
We extend Deep Feedback Control (DFC), recent adaptive control theory framework for error- based learning [24), to a biologically plausible dis-inhibitory microcircuit motif.
We show how a Hebbian plasticity rule with an explicit inhibition dependence can naturally decode credit signals from this microcireuit.
We demonstrate that this rule enables error-based learning in hierarchical networks, naturally stabilizes runaway Hebbian plasticity, and resembles phenomenological plasticity rules under simulated experimental conditions.
Finally, we demonstrate that our learning rule performs comparable to BP in deep neural networks trained on computer vision benchmarks.
Background and previous work
In this article we strive to reconcile phenomenological plasticity models constrained by experiments and models of biologically plausible credit assignment based on normative theories of gradient- learning. In the following we review essential literature of both approaches.

---

2.1 Phenomenological synaptic plasticity models
There is widespread experimental support for the notion of Hebbian synaptic plasticity in the brain, captured in the form of classical long-term potentiation (LTP) and long- term depression (LTD) [2] or brike-tining-dependdee
plasticity (STDP) [3]. An extensive mathematical model catalog captures the phenomenology of these findings [4-8]. Common to most of the above models is that postsynaptic quantities defie plasticity threshold which separates LTD from L.TP induction, which effectively determines the sign of the synaptic weight change [32-34] (Fig. Ia). A plethora of phenomenological plasticity models exist that capture such dependence on firing rate [6], voltage [7], and postsynaptic calcium concentration [5, 8]. For isolated neurons, classic work has demonstrated that Hebbian plasticity can extract principal components from structured data [35] or capture receptive field formation observed experimentally [36]. However, these models do not extend to deep hierarchical networks, nor can they account for the modulation of plasticity through local error signals required for solving the credit assignment problem. Thus, the mechanisms by which synaptic plasticity observed under experimental conditions could give rise to coordinated learning at the circuit- and network level remain elusive.
2.2 Models of biologically plausible credit assignment
The above models are contrasted by normative rules derived from gradient-based learning principles, which often aim at approximating BP [15] (Fig. 1b-c). A crucial aspect of BP is the separation of forward- and backward signaling, which algorithmically separates credit signaling from neuronal ac- tivity [37]. This separation presents a significant challenge for biologically plausible implementations. as it presumes a distinct separation through either learning phases or separate pathways.
Equilibrium propagation (EP) offers one possible, if only partial, solution to this dilemma. EP posits that local errors are derived as variations in neuronal activity at a network equilibrium state [29]. Yet, classic EP still requires separate processing phases for each input, inconsistent with neurobiology. However, recent work suggests possible ways of alleviating the requirement for distinct phases through neural oscillations [38].
An alternative to separate phases for forward and backward passes is to separate them spatially, Predictive coding models [19, 25, 39] exemplify this idea, whereby errors are computed in dedicated neuron-speciff error units by comparing each neuron's activity to a top-down prediction. Recent work has suggested that the electrotonically segregated apical dendrites of cortical pyramidal neurons [37, 40] could represent local errors in learning [21, 22).
However, to approximate BP, a common theme across these models is their dependence on explicit error-modulation of plasticity [30] (Fig. Ib-c). While mrodulmdddlteed learning rules prove func- tionally useful and, in some cases, can match the performance of BP, experimental evidence for their existence inconelusive.
particular, they fall short of capturing established properties of experimentally observed plasticity, such threshold between LTD and LTP that is governed by postsynaptic activity [32-34, 41].
Finally, Payeur et al. [23] proposed a unique multiplexing approach by encoding forward and feedback signals in the event and burst rate of output spike trains. Such burst-dependent plasticity rules capture essential facets of phenomenological plasticity [31] (Supplementary Fig. S1; see Appendix A). However, the model primarily applies to cortical Layer pyramidal cells with electrically segregated dendritic trees. In contrast, it may not work for layer 2/3 neurons or other brain areas without segregated dendrites.
Since most of the above models focused on approximating BP, they face another potential issue: they usually require weak feedback, which causes only slight perturbation, or nudge, of the input-driven equilibrium. This requirement contrasts significantly with a wealth of experimental literature suggesting that feedback connections in the brain substantially influence neuronal activity [42-44]. Rather than striving to find biologically plausible separations between forward and backward signaling, recent modeling studies used an approach grounded in adaptive control theory [25, 45 48). These models leverage strong feedback signals to steer neuronal activity to align with a given target output. While strong feedback aligns more closely with neurobiological observations, these models are also dependent on explicitly eroor-modulated learning rules, wherein the error is computed from either the difference between the controlled and uncontrolled states of neuronal activity similar to EP or the difference between activity in segregated neuronal compartments, as in dendritic error coding.
3

---

Still, it remains to be seen how such learning could be implemented at the circuit level with plasticity rules that capture experimental findings. this article, we propose putative circuit-level solution to this issue by mapping the notion of feedback control onto a known dis-inhibitory circuit motif and combining it with an inhibition-modulle Hebbian plasticity rule.
3
Model
To study whether biological microcircuits could naturally interpret dis-inhibitory feedback signals local errors control the sign of synaptic plasticity, consider a continuous time dynamic multi-layer network comprised of excitatory and inhibitory neurons in each layer with dis-inhibitory feedback connections.
3.1 Neuronal dynamics
The membrane potential dynamics of the excitatory and inhibitory neurons in layer are described by the following ordinary differential equations (ODEs):
TB u (t) W (t) (t)
(1)
d
7 di u -u!((( + (t) Q.c(()
with W, the afferent synaptic weights from the previous layer and r, ó(u,) a smooth, monotoni-
cally increasing nonlinear activation function. Note that here we made the simplifying assumption that each excitatory neuron has an associated inhibitory neuron and that both are connected locally within microcircuit. For all simulations, we use the soft rectifying nonlinearity Φ(u) 33 log(1 +exp(u-7)) which | and y are parameters controlling the scale and shift the activation function, respectively, Dis-inhibitory feedback is mediated through top-down control signals c(t) relayed to each layer i through an associated feedback weight matrix Q; (Fig. 2). The input layer ro()) data-dependent and not influenced by top-down feedback. For constant input current the network dynamics settle to the equilibrium state
6 W 1-1
F!
= #
Q
Figure 2: Illustration of multi-layer network with dis-inhibitory control microcircuits (left). Each network unit consists of an excitatory and inhibitory neuron that are recurrently connected (right). The top-down control signal to each layer Qze(I) is relayed through dis-inhibitory afferents (orange).
(x)))
p (t)
O0O
(t)
Que(()
Q
Qu
W.
W
W
3.2
Feedback control
As in previous work DFC [24, 47], our model uses feedback control to drive the output activity of the network towards the target activity r by minimizing the magnitude the output error:
ett) aC(rL, r
drr
(t)
where L(rL, r is label-dependen supervised loss function defined on the network's 's output
activity. For a simple mean squared error (MSE) loss, L 1/n Σ., 1/2ll1 - r (n))) the error for each datapoint is directly related the difference between the network's output and the target, specifically e(t) r (t).).
Leaky proportionallintgres controller. In our model, we leaky proportional-inteerall controller to compute the feedback control signals c(t):
Te d di e(t) - cit dt
c(t) ktt + ki
(5)
where kp and k are the proportional and integral control constants, respectively.

---

Dis-inhibitory feedback connectivity. Next, we have to connect the controller to the controlled quantities. In the context of our model, we assume that control signals are relayed via inhibitory interneurons. We thus have to specify the feedback weights connecting the control signals to the local microcircuits defined in the previous section. While we make no claims about how suitable control feedback is generated in neurobiology. in this article we merely assume that suitable control signals exist and that they are mediated via inhibitory interneurons. To that end, the feedback weights Q: need to be chosen such that the output loss C(r,, ry.") is minimized at the controlled equilibrium state. To fulfill this requirement, the column space of the concatenated feedback weights of the network, Q _ ....... O must be equal to the row space of the network Jacobian J at steady- state [24]. This Jacobian characterizes how infinitesimal perturbations of each controlled quantity, e.g.. neuronal activation, relates to changes in the network's 's output rz. In contrast to previous models relying on top-down control, our network consists of recurrently connected excitatory and inhibitory units whilst top- imput is targeting the inhibitory population exclusively. Since want to model control through dis inhibitiyy circuits, we assume that the Jacobian for the controller is defined with respect the inhibitory membrane potential each layer:
= rrr Ōr
J J1
Jzl
(6)
Oul
Out,
where we use respect to the vector ut. It has been shown that wide range of possible feedback weights Q can match the row space of J in the DFC framework. One simple way of ensuring this condition is met is to set the feedback weights at each layer proportional to the transposed Jacobian, i.e., Q, = J However, the Jacobian depends on the input and the neuronal activity. It is thus changing over time until an equilibrium state is reached. To avoid changing feedback weights over time for given input, we compute them based on the network Jacobian at the uncontrolled equilibrium state with c = 0
drg V Vulur denote the Jacobian matrix of partial derivatives of the vector ry with au!
ÔrL au!
T
J²' =
-/=0
with üi! corresponding to the inhibitory membrane potentials in Layer at the uncontrolled equilibrium state.
Learning as minimization of control dis-inhibitory neuronal circuits. Given suitable feedback weights and a strong influence on the network activity by the controller, neuronal activity can change considerably compared to the uncontrolled steady-state. Tracing the steps of [47], learning rules can be derived from minimization of control objective
H
1 2
Σ
(
where & (n) is the steady-state feedback control signal for datapoint n. Formally, it can be shown that minimizing the above surrogate loss H also minimizes the output loss (see Appendix B.I). We start with the following learning rule which minimizes H::
d Tw Wi dt
postsynaptic
presynaptic
where d'(u) is the derivative of the activation function and @ denotes element- wise multiplication.
The sign of the weight change is determined by the error projected onto each neuron by the feedback controller at the equilibrium. which following Eq. (3) is encoded as Q = F. ä.. While Eq, (9) minimizes H. when provided with sensible feedback signals, it does not constitute a local learning rule because it explicitly depends on the inhibitory membrane potentials å! which excitatory neurons cannot access directly (cf. Fig. 1).
3.3
A Hebbian learning rule for error-modulated learning through dis-inhibitory control
We were wondering whether excitatory neurons could compute an effective local approximation of Eq. (9) by estimating the inhibitory membrane potentials from locally available quantities. To that
5

---

end, we first we-write Eq. (9) as
W;- -- (f)) -
o' (uf))
F (F=_))"
(10)
di
where we substituted the inhibitory membrane potentials ût with the inverse activation function 6- (f!). Mathematically, Eqs. (9) and (10) are equivalent, but conceptually, it re-frmmes the problem of non-locality since it would require an excitatory neuron to compute the inverse activation function from the recurrent inhibitory current, which is locally available. While it is hard to imagine how neurons would invert the activation function of other neurons exactly, we assume that they could conceivably compute linear approximation, leading to learning rule of the following general form:
d
( - - o Φ' (ū
W; TW dt
(11)
Here the parameter 0, takes the role of a postsynaptic plasticity threshold, common to many phe- nomenological plasticity models [6-8], while 6, adds an inhibitory current dependence to this threshold. In practice, we obtain 8, and 8, through first-order Taylor expansion of the inverse activation function around a given linearization point (see Appendix B). In the next sections, we will see that, depending on the inhibitory activation function and the linearization parameters, the model reconciles aspects of phenomenological plasticity with an effective mroo-mooulation mechanism as demanded by normative theories of gradient-based learning.
4 Learning with dis-inhibitory control accounts for key plasticity experiments
Most experiments on synaptic plasticity are performed in vitro under highly controlled conditions, in which pairs of connected neurons are isolated. This enables researchers to investigate the plasticity at single synapses in the absence of interfering activity from the local microcircuit or long-range synaptic afferent connections. Such experimental conditions would likely interfere with any putative error- mroor-modulation of synaptic plasticity since long-range synaptic afferents are either severed during sample preparation or do not transmit plausible activity levels. To account for such possible experi- mental confounding factors and to compare experimentally observed Hebbian plasticity, we probe error-modulated learning rule in three different settings: full microcircuit with "closed-loop feed- back", intact "microcircuit without feedback"" and isolated neurons resembling in-vivo experimental conditions with "direct control of inhibition" (Fig. 3). In each setting, vary the excitatory input to the excitatory neuron and calculate the resulting weight change as function of the postsynaptic firing rate using Eq. (1I). For a qualitative comparison to previously suggested error-driven plasticity rules, we evaluated learning rules using dendritic error coding [22, 24, 30] or burst-dependent plasticity [23, 31] in comparable settings (Supplementary Fig. S1; see Appendix A).
iis-inhibition controls the sign of plasticity in the intact microcireuit. We first considered a simplified version of the intact microcircuit with top-down dis-inhibitory feedback, for which our learning rule was derived. In this circuit, top-down control drives neuronal activity towards target value tar We computed the weight updates dictated by our learning rule for two different targets as a function of the neuronal firing rate. In this setting, the learning rule exhibits two stable fixed points separated by an unstable one (Fig. 3a). Importantly, stable fixed point exists at the the target firing rate, i.e. , = , (Fig. 3a). At this fixed point, the sign of plasticity is determined by the top-down controller through the error decoded by the learning rule (11). This behavior is in line with the behavior of learning rules with explicit modol-modllaioon in this simplified setting (cf. Fig. lb, Supplementary Fig. S1).
Moreover, in contrast to purely error-modulated plasticity rules, our rule exhibits an LTD region flanked by a stable fixed point in neuronal firing rates at zero and an unstable fixed point at intermediate firing rates. The existence of this LTD regime is a direct consequence of the local approximation used in its derivation (cf. Eq. (11)) and depends on the chosen parameters of the linearization (Supplementary Fig. S2; see Appendix B). For proper error- modulated learning in our framework, we have to ensure that each neuron's 's activity does not exclusively stay in this region over time and across different inputs. In neurobiology, this activity regime could, for instance, be attained through homeostatic plasticity [51]. Thus our learning rule exhibits modulated plasticity at the upper fixed point when embedded in an intact microcireuit with top-down feedback.

---

a)
control
b)
d)
Closed-loop feedback
Microcireuit without feedback
Direct control of Inhibition
Experiments Isolated neuron
2.0
porr = 4.0
20
- 3.0
1.5
Inh. firing rate
[%]
AW
1.0
0.5
-20
4
10*
Firing rate r'(a.u)
Firing rate r[a.u.]]
Firing Ala.u.]
Stim. frequency [Hz]
Figure A Hebbian learning rule for mrdor-modulated learning through dis-inhibitory control resembles plasticity observed in single-neuron clectrophysiology experiments. (a) Weight change AW of a single synapse as a function of postsynaptic firing rate rp. Different colored lines indicate two different postsynaptic firing rate targets p indicated by colored arrows. The top-down feedback onto the interneuron is proportional to the error tor , In the intact microcircuit with closed-loop modulated learning. (b) Same as (a), but with
feedback, our learning rule naturally leads top-down connections ablated. The two shades of green represent two different linear approximations of the inverse inhibitory activation function (see inset; inverse activation function black). The plasticity rule resembles multi-stable Hebbian plasticity rule [49]. (c) Same as before, but for an isolated neuron without microcireuit. Different shades of blue correspond to different amounts of inhibitory current. Different levels of injected inhibitory current lead to differnt values for the plasticity threshold, but the stable fixed point disappears in the absence of recurrent inhibition. (d) Experimentally observed plasticity with activity-dependent LTD and LTP redrawn from [50]. The data qualitatively resembles our learning rule the open-loop setting (cf. panel (c)).
Plasticity in isolated microcircuit is self-stabilizing. To examine how our learning rule behaves in the absence of top-down control signals, we investigated plasticity dynamics in an isolated microcireuit without control feedback, while local circuit connectivity between excitatory and inhibitory neurons was left intact. In this setting, the inhibitory membrane potential does not encode an error signal that be decoded by the learning rule. While explicitly error-modulated learning rules would not exhibit any synaptic weight change in this setting, our Hebbian learning rule predicts weight changes due to the imperfect approximation of the inverse function. Fig. 3b depicts the resulting plasticity dynamics for two different linear approximations. Embedded in a local microcircuit, the plasticity rule still exhibits both an LTD and LTP regime and stable fixed point that depends on the learning rule parameters. Notably, the plasticity rule embedded in an isolated, but intact microcireuit is self-stabilizing through recurrent inhibition. Interestingly, the necessity of such stable fixed point at higher activity levels for stable learning has been postulated previously in theoretical work [49, 52]. As before, the presence and location of the stable fixed point depend on choice of plasticity parameters (Supplementary Fig. S2).
Plasticity induction changes under direct control of inhibition. Experiments on excitatory plasticity in vitro are commonly performed under conditions designed minimize the interference of inhibitory activity, for example by applying GABA antagonists [53]. To study plasticity induction in our model under such simulated experimental conditions, we blocked recurrent connections from excitatory to inhibitory neurons, so that inhibitory activity is independent of excitatory activity. Additionally, we controlled inhibitory activity, as could be achieved, for instance, through current injection optogenetic manipulations in experiments. In the absence of any inhibitory activity, ie. = 0, our learning rule reduces to the form Aw & Tpre (Tpost - 0) o' (rpost and loses its stable fixed point (Fig. 3c). Thus, in the absence of inhibition, the weight update prescribed by our plasticity model resembles Hebbian plasticity rules commonly observed under experimental conditions (Fig. 3d). However, our model predicts that direct control over inhibitory inputs to the excitatory neuron should shift the postsynaptic plasticity threshold to larger values.
In summary, dis-inhibitory control accounts for key plasticity experiments while also supporting error-driven learning top-down controlled microcireuits. Additionally, self-stabilizing ca- pabilities provide possible explanation to why in vitro experiments have failed to uncover
7

---

a)
<00
b)
Before learning
Exact inverse
4
Linear threshald
Output
0.4
(x)
000
2
0.2
Wx
0.0
OOOO
5
Wi
After learning
75
Beckk
0.4
50
Oureo
0.2
0.0
Time 1s]
1
2
3
4
2
4
Time [s]
Time [min]
Figure 4: Online learning using dis-inhibitory control of Hebbian plasticity student-teacher task. (a) A teacher network (left) of size 30-20-2 implements a nonlinear mapping from an array of input sine waves x(t) (bottom) to an output target fx(()). student network (right) of the same size learns to approximate the teacher function by minimizing the control signal Q,t(t) of dis-inhibitory feedback controller at each layer. (b) One example teacher output neuron (black) and the corresponding student neuron (teal) before and after learning. The yellow bar indicates when feedback control is active. (c) MSE loss L and least control loss H over time for student networks trained with the exact update rule derived in Eq. (10) and the linear threshold rule (Eq. (11)).
stabilizing mechanism for excitatory synaptic plasticity. Next, we test whether our learning rule allows hierarchical networks solve nonlinear function approximation problems.
Dis-inhibitory control orchestrates learning in multi-layer networks
To explore our model's ability to train multi-layer networks, we first designed simple continuous- time -ineennnniooaal student-teacher learning task (Fig. 4a; see Appendix C for details). In this task, a randomly initialized teacher network with fixed parameters ((((()) is given set of 20 sine wave inputs with randomly chosen amplitudes, frequencies, and phases x(t). An architecturally identical student network but with different initial weights receives the same input x(t) and is tasked to reproduce the teacher's output, i.c. rt. = f(x(t)) evaluated through an MSE loss function. A dis-inhibitory feedback controller as described in the previous section is continually driving the student network's activity towards lower loss real time. We trained the student network with either the exact non-linear inhibitory threshold rule Eq. (10) the Hebbian learning rule with linear inhibitory threshold Eq. (11). Neuronal and weight dynamics were simulated in continuous time using explicit 5th order Runge-Ken Kutta method. To make sure that the feedback controller is aligned with the changing Jacobian during learning, the feedback weights were plastic and continuously evolving towards the average network Jacobian (see Appendix C).
Before learning, the student network did not follow the target closely in the open-loop setting, i.e., when the control signal was turned off. However, as soon as dis-inhibitory control was activated, the output activity closely followed the target (Fig. 4b). We then trained the student network for total of 300 seconds of continuous sine wave inputs using either Eq. (10) or Eq. (11). After learning, the student network output closely followed the target in the open-loop setting, accompanied by substantial reduction of the open-loop MSE loss (c(t)
0) and the surrogate loss H. over the course of training (Fig. 4c). Finally, we observed that the linear threshold learning rule resulted in comparable performance to the exact inverse learning rule. Thus, an inhibitory modulated Hebbian learning rule is capable of solving nonlinear learning task when credit is relayed through local dis dis-inhibitory microcireuits acting feedback controller.
5.1 Training multi-layer networks on classification tasks through dis-inhibitory control
Having confirmed that our learning rule capable of error-driven learning through minimization of control on a simple continuous-tima learning task, we wondered whether we could train a multi-layer perceptron (MIPP) on standard image classification tasks. To that end, we implemented a MI.P with excitatory-nnhiiiory microcireuit units. Networks were comprised of either one three hidden layers
8

---

and used parameterized soft rectifier activation function for all units. Numerical integration was performed using fifth-order Runge-Kutta method with an adaptive step size to allow the network to reach an equilibrium state for each input (see Appendix C for details). Networks were trained either with the exact inverse rule in Eq. (10), or the linear threshold rule (Eq. (11)). To make training more robust and less dependent on initialization, the linear approximation of the inverse function was obtained through first-order Taylor expansion around a secuonn-ppccffie parameter I; which tracked the average inhibitory firing rate across each batch (see Appendix C). With these settings we trained the networks on MNIST [54] and Beshion-MNISTS [55]. For comparison, we also trained conventional MLPs with the same neuron numbers but strictly feed-forward connectivity using BP.
Table I:
Test accuracy in % for networks trained with BP or dis-inhibitory feedback control. Reported values are mean dtsdde (n 10). For validation accuracy see Appendix C.
MNIST
Eishion-MNISS
Number of hidden layers
3
3
Backprop
98.1 + 0.2
98.3 + 0.1
89.3 0.3
89.4 + 0.2
Exact inverse
97.7 + 0.2
98.0 0.2
89.1 + 0.2
89.1 + 0.2
Dis-inhibitory
Linear threshold
97.1 + 0.1
96.5 + 0.1
87.6 + 0.4
86.8 + 0.2
control
Exact inverse (Avg J)
97.8 0.1
97.0 + 1.5
88.7 1.8
88.2 + 0.5
Linear threshold (Avg J)
96.5 + 0.1
96.0 + 0.3
86.4 + 0.3
84.6 + 0.3
We observed that networks trained using dis- dis-inhibitory, control with the exact inverse learning rule performed almost on par with standard BP on both datasets (Table 1). Using the linear threshold rule led to a slight drop in accuracy in all cases. It is possible that this gap could be narrowed further by choosing different linearization parameters F; since the performance of the exact inverse rule suggests that improving the approximation of the error translates to higher accuracy.
In above simulations, the Jacobian used for caleulating the feedback signals is input-dependent and feedback weights are thus different for each stimulus. In neurobiology, feedback would presumably be relayed via synaptic connections that do not change this rapidly. To test whether successful learning is possible with more slowly varying feedback weights we repeated the above simulations with feedback weights that slowly tracked the average Jacobian (see Appendix C). This change did not compromise learning, although resulted in a small but noticeable drop in accuracy compared to the ideal data-dependent Jacobian (Table 1). In summary, the combination of dis-inhibitory control with a Hebbian learning rule with an inhibition-dependent threshold allows training MLPs on vision datasets such MNIST and Fashion-MNIST to accuracy values close to networks trained with BP.
6 Discussion
In this article, we introduced a Hebbian learning rule with an inhibitionndeepeddett threshold, which, through dis-inhibitory microcircuit dynamics, allows top-down feedback to control the sign of plasticity. Notably, the learning rule captures essential aspects of classic phenomenological Hebbian plasticity models under simulated experimental conditions disrupting recurrent inhibition in the local microcircuit. In contrast to standard Hebbian plasticity models, our model is stable when recurrent inhibition is intact without requiring additional homeostatic or compensatory mechanisms. Finally, we show how dis-inhibitory control is sufficient to train MLPs on vision tasks with performance levels close to classical BP.
Dis-inhibitory control reconciles error-based learning with classic Hebbian plasticity models. The learning rule we put forward in this article has a postsynaptic plasticity threshold 0 (cf. Eq. (1I)), a neuron- neuron-specific parameter potentially subject to its own temporal dynamics. This threshold makes reminiscent of the BCM rule [36]. Moreover, our model extends classic plasticity models by an explicit dependence on inhibition, which adds dynamic, rapidly evolving component to the plasticity threshold. We derived this -ual-threshold mechanism within a normative minimization-of-com objective. Its functional role is to decode credit signal, relayed through dis-inhibitory afferents, from changes the inhibitory current. Notably, the inhibitory threshold in our model confers an additional helpful property in that it induces rapid compensatory plasticity mechanism that prevents

---

pathological runaway LTP, usually associated with Hebbian plasticity, through recurrent inhibition and in the absence of top-down control signals or any additional homeostatic mechanisms. Theoretical studies have argued that such stabilizing mechanism for high firing rates should exist to counteract the runaway potentiation of Hebbian plasticity [49, 52].
Experimentally testable predictions. Our model puts forth several testable predictions. First, we anticipate a direct modulation of plasticity induction through inhibitory currents in conventional excitatory long-term plasticity induction protocols. We predict that postsynaptic plasticity thresholds [32, 33] should be influenced by inhibitory current injection, even when the postsynaptic activity is kept constant in experiments. Most plasticity experiments proceed in the presence of GABA antagonists or sodium channel blockers, which may obscure the direct impact of inhibition on the plasticity threshold. Nevertheless, abundant experimental evidence supports the idea that inhibition influences classic induction protocols at excitatory synapses [56-59] and that GABAergic afferents can switch the sign of plasticity [60, 61]. Second, our model suggests that blocking dis-inhibitory circuits during an error-based learning paradigm should block or influence learning. Consistent with this hypothesis. dis-inhibitory microcircuits have been implicated with the gating of plasticity and behaviorally relevant learning in the Amygdala [62, 63]. Hippocampus [64], and sensory cortices [65-69] (for a review, see [70]. Conversely, activating the same circuitry should affect or trigger learning during specific tasks.
Limitations. Several limitations should be considered when interpreting this study's results. First, while our learning rule minimizes the loss, it does not necessarily follow the negative gradient. This difference could lead to sub-optimal learning dynamics and we will explore its impact in future work.
Moreover, our circuit model requires specific one-to-one connectivity between excitatory and in- hibitory interneurons that is inconsistent with circuit motifs observed in the brain. In biological microcircuits, excitatory neurons usually exceed the number of inhibitory neurons [71], and inhibitory interneurons typically provide inputs to many local excitatory cells and vice versa. Nevertheless, may be possible consider our model's 's inhibitory threshold (cf. Eq. (11)) as a local inhibitory current derived from multiple presynaptic targets. In this scenario, the challenge for excitatory neurons is to estimate their respective contribution to the inhibitory current they receive. In future work, we will explore the possibility of learning an estimate of the expected inhibitory current by implementing inhibitory plasticity to achieve an excitatory iiniiiiirroiiiiortyyyoo balanced state [72] and its co-dependence with excitatory plasticity [73]. Inhibitory plasticity and its interactions with excitatory plasticity is supported by experiments [74-76] and has been the focus of recent computational models [77, 78]. However, neither of these studies explored the functional relevance of co-dependent plasticity for credit assignment and circuit- level learning.
Finally, the present model does not adhere to Dale's 's law in synaptic connections between hidden layers or the feedback pathway. Instead, we integrated the local population of dis-inhibitory interneurons into the dynamics of top-down controller that modifies inhibitory activity bidirectionally. Biologically, this could be achieved through high baseline firing rates of dis-inhibitory interneurons or top-down feedback connections that target inhibitory interneurons directly [42]. In future work, we will explore detailed cortical architectures for dis-inhibitory feedback controllers that allow bi-directional modulation.
In summary, we made the first step to reconcile realistic circuit models and phenomenological plasticity rules with normative theories relying on error-modulated plasticity to solve the credit assignment problem. Specifically, we showed how top-down feedback signals targeting specific interneurons could efficiently modulate neuronal activity and plasticity. Our results highlight the potential of learning algorithms beyond BP, showcase their ability to incorporate diverse plasticity phenomena observed in neurobiology, and open the door for exciting future research.
Acknowledgments
We thank all members of the Zenke Group for valuable comments and discussions. This project was supported by the Swiss National Science Foundation [grant number CCEFP3_202981 and the Novartis Research Foundation.
10

---

References
[1] DO Hebb. The organization of behavior; a neuropsychological theory. The organization of behavior; a neuropsychological theory. Wiley, Oxford, England, 1949.
[2] Robert C Malenka and Mark F Bear. LTP and LTD: an embarrassment of riches. Neuron, 44(1): 5-21, 2004. doi: mw0.:/..eeroon2004....999
[3] Daniel E Feldman. The spike-timing tinkee dependence of plasticity. Neuron, 75(4):556-571, 2012. doi: 10.1016/j ne..016/j.euuron... www..e...o.......00... 2012.08.00....
[4] Wulfram Gerstner and Werner M Kistler. Mathematical formulations of hebbian learning. Biological cybernetics, 87(5-6):404-415, 2002. doi: 10.1007/50042--0--05553y
[5] Harel Z. Shouval, Mark F. Bear, and Leon N. Cooper. A Unified Model of NMDA Receptor- Dependent Bidirectional Synaptic Plasticity. Proc Narl Acad Sci U S A, 99(16):10831-00833 2002. doi: 10. 10.1073/pnas. 00.1073/pnas.1552440999 152343099.
[6] Jean-Pascal Pfister and Wulfram Gerstner. Triplets of Spikes in Model of Spike Timing- Dependent Plasticity. Neurosci, 26(38)9973-9682,, 2006. doi: 10.1 10.1523/INUUROCI. 1425-06.2006.
[7] Claudia Clopath, Lars Büsing, Eleni Vasilaki, and Wulfram Gerstner. Connectivity reflects coding: model of voltage-basee STDP with homeostasis. Nat Neurosci, 13(3):344-333 352, 2010. doi: 10.1038/nn.2479.
[8] Michael Graupner and Nicolas Brunel. Calcium-based plasticity model explains sensitivity of synaptie changes to spike pattern, rate, and dendritic location. Proc Nail Acad Sei U A, 109 (10):3991-3996, 2012.
doi: 10.1073/pnas. 109359100.
[9] Nicolas Frémaux and Wulfram Gerstner. Neuromodulated Teet----mii-eeeeeee plasticity, and theory of Three-Factor learning rules. Frontiers in neural circuits, 9:85-85, 2016. doi: 10.3389ffncir2215500033.
[10] Eukasz Kuśmierz, Takuya Isomura, and Taro Toyoizumi. Learning with three factors: modulat- ing hebbian plasticity with errors. Current opinion in neurobiology, 46:1 17-1777 2017. doi: 10.1016/j.comb.20177
[11] Verena Pawlak, Jeffery R Wickens, Alfredo Kirkwood, and Jason N D Kerr. Timing is not Everything: Neuromodulation Opens the STDP Gate. Front Synaptic Neurosci, 2:146, 2010. doi: 10.3389/fnsym.2200001146
[12] Zuzanna Brzosko, Susanna B Mierau, and Ole Paulsen. Neuromodulation of Dependent plasticity: Past, present, and future. Neuron, 103(4):563-581,))) 2019. doi: 10.1016/j. neuron. w0w.oo..100........0
[13] Justin Werfel, Xiaohui Xie, and H Sebastian Seung. Learning curves for stochastic gradient descent in linear feedforward networks. Neural computation, 17(12):2699-22 27118, 2005. doi: 10.1 10.166//0997700777733333
[14] Timothy P Lillicrap. Adam Santoro, Luke Marris, Colin J Akerman, and Geoffrey Hin- ton. Backpropagation and the brain. Nature reviews. Nearoscience, 2020. doi: 10.1038/ 041583-020-0277-3..
[15] David E Rumelhart, Geoffrey E Hinton, and Ronald Williams. Learning representations by back propaaaroggggiggggg errors. Nature, 323(6088) 1986. doi: 10. 10.1003/323533a0.
[16] Crick. The recent excitement about neural networks. Nature, 337(6203): 12000)3::22211000 121-11922222 1989. doi:
10. 10.1003//33 1011103377999
[17] Pieter Roelfsema and Anthony Holtmaat. Control of synaptic plasticity in deep cortical networks. Nature reviews. Neuroscience, 19(3):166-180 2018. doi: 10.1038/nm.2018.66

---

[18] Blake A Richards, Timothy P Lillicrap. Philippe Beaudoin, Yoshua Bengio, Rafal Bogacz, Amelia Christensen, Claudia Clopath, Rui Ponte Costa, Archy de Berker, Surya Ganguli, Colleen Gillon, Danijar Hafner, Adam Kepecs, Nikolaus Kriegeskorte, Peter Latham, Grace W Lindsay, Kenneth Miller, Richard Naud, Christopher Pack, Panayiota Poirazi, Pieter Roelfsema, João Sacramento, Andrew Saxe, Benjamin Scellier, Anna C Schapiro, Walter Senn, Greg Wayne, Daniel Yamins, Friedemann Zenke, Joel Zylberberg, Denis Therien, and Konrad Kording, A deep learning framework for neuroscience. Nature neuroscience, 22((1)) 2019. doi: 10. 1..1008//41 00.1038/44153-019--0000 593-019-0520-2.
[19] James C R Whittington and Rafal Bogacz. An approximation of the error backpropagation algo- rithm in a predictive coding network with local hebbian synaptic plasticity. Neural computation, 1229-122222 2017. doi: 10. 1,2///0OOOO Ll_099
[20] James C R Whittington and Rafal Bogacz. Theories of error Pack-Propagation in the brain. Trends in cognitive sciences, 23(3):235-255 2019. doi: 10.1016/j.tics.211 2011..iii..21....... 8.10005
[21] Jordan Guerguiev, Timothy P LLillicrap. and Blake A Richards. Towards deep learning with segregated dendrites. eliife, 6, 2017. doi: (1.755//el.ffe.29900
[22] João Sacramento, Rui Ponte Costa, Yoshua Bengio, and Walter Senn. Dendritic cortical miero- circuits approximate the backpropagation algorithm. In S Bengio, H Wallach, H Larochelle, K Grauman, N Cesa-Biancho and Garnett, editors, Advances in Neural Information Process- ing Systems 31, pages 8721-8732. Curran Associates, Inc., 2018
[23] Alexandre Payeur, Jordan Guerguiev, Friedemann Zenke, Blake A Richards, and Richard Naud. Burst-dependent synaptic plasticity can coordinate learning in hierarchical circuits. Nature neuroscience, 2021. doi: 10. 1033//41593-021-00833.
[24] Alexander Meulemans, Matilde Tristany Farinha, Javier Garcia Ordonez, Pau Vil- imelis Aceituno, João Sacramento, and Benjamin F. Grewe. Credit assignment in neural networks through deep feedback control. In M. Ranzato, A. Beygelzimer, Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 4674-4687. Curran Associates, Inc., 2021.
[25] Yuhang Song, Beren Millidge, Tommaso Salvatori, Thomas Lukasiewicz, Zhenghua Xu, and Rafal Bogacz. Inferring neural activity before plasticity: A foundation for learning beyond backpropagation. bioRxiv, 2022. doi: 10.1101/202.005. 17.492325.
[26] Blake A. Richards and Timothy P. Lillicrap. Can neocortical feedback alter the sign of plasticity? Nat Rev Neurosci, 2018. doi: 15800///1113-011----1
[27] Carsten K. Pfeffer, Mingshan Xue, Miao He, Z. Josh Huang, and Massimo Scanziani. Inhibition of inhibition in visual cortex: the logic of connections between molecularly distinct interneurons. Nature Neuroscience, 1668 2013. doi: 10. 10.8n///n..3
[28] Adam Kepecs and Gordon Fishell. Interneuron cell types are fit to function. Nature, 505(7483): 318-326, 2014. doi: 10. 1038/nature 12983.
[29] Benjamin Scellier and Yoshua Bengio. Equilibrium propagation: Bridging the gap between energy-based models and backpropagation. Frontiers in computational neuroscience, 11:24, 2017. doi: 10.3389/fncom..001 2017. .00024.
[30] Robert Urbanczik and Walter Senn. Learning by the dendritic prediction of somatic spiking. 2013..
Neuron, 81(3))521-528, 2014. doi:
[31] Will Greedy, Heng Wei Zhu, Joseph Pemberton, Jack Mellor, and Rui Ponte Costa. Single-phase deep learning in cortico-cortical networks. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh. editors, Advances in Neural Information Processing Systems, volume 35, pages 24213-24225, Curran Associates, Inc., 2022.
[32] PJ Sjöström, G G Turrigiano, and S B Nelson. Rate, timing, and cooperativity jointly determine cortical synaptic plasticity. Neuron, 32(6): 149-1 164, 2001. doi: 10. 10.1016640896-6273(00) 00542-6.
12

---

[33] A Artola, S Bröcher, and W Singer. Different gettage-dependeen thresholds for inducing long- term depression and long-term potentiation in slices of rat visual cortex. Nature, 347(6288): 69-72, 1990. doi: 10. 10.1038/347069a.
[34] Sukbin Lim, Jillian L McKee, Luke Woloszyn, Yali Amit, David J Freedman, David Sheinberg, and Nicolas Brunel. Inferring learning rules from distributions of firing rates in cortical neurons. Nature neuroscience, 18(12): 1810, 2015. doi: 10. 10388/n..41.8..
[35] Oja. A simplified neuron model as a principal component analyzer. Journal of mathematical biology, 15(3):267-273, 1982. doi: 10. 10.10077b1000256877.
[36] L Bienenstock, L Cooper, and P W Munro. Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex. The Journal of neuroscience, 2(1):32-48, 1982. doi: 1530NNEUROS1100000000002000000
[37] Blake A Richards and Timothy P Lillicrap. Dendritic solutions the credit assignment problem. Current opinion in neurobiology, 54:28-36, 2019. doi: 10.1016/j.conb.200 www.oooooooo....oooo. 8.08.033.
[38] Axel Laborieux and Friedemann Zenke. Holomorphic equilibrium propagation computes exact gradients through finite size oscillations. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances Neural Information Processing Systems, volume 35, pages 22950-12963 Curran Associates, Inc., 2022.
[39] R P Rao and D H Ballard. Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nature neuroscience, 2(1):79-87, 1999. doi: 10. 1038/4585...
[40] M E Larkum, J J Zhu, and B Sakmann. new cellular mechanism for coupling inputs arriving at different cortical layers. Nature, 398(6555) 341, 1999. doi: (0.1038/1866)
[41] C M Coussens, D Kerr, and W Abraham. Glucocorticoid receptor activation lowers the threshold for AMDDA pus through activation of voltage-dependent calcium channels. Journal of neurophysiology, 78 (1):1-9, 1997. doi: 10. 10.1152jjn.1997778.111
homosynaptic long-term depression in the hippocam-
[42] Shan Shen, Xiaolong Jiang, Federico Scala, Jiakun Fu, Paul Fahey, Dmitry Kobak, Zhenghuan Tan, Na Zhou, Jacob Reimer, Fabian Sinz, and Andreas S Tolias. Distinct organization of two cortico-cortical feedback pathways. Nature communications, 13(1):6389, 2022. doi:
10.1038844146--022233888
[43] Andreas J Keller, Morgane M Roth, and Massimo Scanziani. Feedback generates second receptive field in neurons the visual cortex. Nature, 582(7813):5455 549, 2020. doi: 10.1038/ 111886020-2319-44
[44] Edward Zagha, Amanda Casale, Robert N S Sachdev, Matthew J McGinley, and David A McCormick. Motor cortex feedback influences sensory processing by modulating network state. Neuron, 79(3):667-577, 2013. doi: n0.00166..curon.2113.6666 208..06.08.
[45] Aditya Gilra and Wulfram Gerstner. Predicting non-linear dynamics by stable local learning in recurrent spiking neural network. eLife, 6:e28295, 2017. doi: 10.7554/cLife.28295
[46] Alireza Alemi, Christian Machens, Sophie Deneve, and Jean-Jacques Slotine. Learn- ing Nonlinear Dynamics in Efficient, Balanced Spiking Networks Using Local Plasticity Rules. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1), 2018. doi: 00.1609/aaii.v32..1111
[47] Alexander Meulemans, Matilde Tristany Farinha, Maria R. Cervera, João Sacramento, and Benjamin Grewe. Minimizing control for credit assignment with strong feedback. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors, Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 15458-15485 PMLR, 2022.
13

---

[48] Pau Vilimelis Aceituno, Matilde Tristany Farinha, Reinhard Loidl, and Benjamin F Grewe. Learning cortical hierarchies with temporal hebbian updates. bioRxiv, 2023. doi: 10.1101/2023. 01 01.02.522459.
[49] Friedemann Zenke, Everton J Agnes, and Wulfram Gerstner. Diverse synaptic plasticity mechanisms orchestrated to form and retrieve memories spiking neural networks. Nuture communications, 6:6922, 2015. doi: 10. 00.108//comms7922.
[50] A Kirkwood, M C Rioult, and M F Bear. Experience-dependdnt modification of synaptic plasticity in visual cortex. Nature, (666))55)5525555 1996. doi: 10. 00.1038/381226a..
[51] Gina Turrigiano. Homeostatic synaptic plasticity: local and global mechanisms for stabilizing neuronal function. Cold Spring Harbor perspectives in biology, 4(1):a005736, 2012. doi:
00110000/shpeespct..0677...
[52] Friedemann Zenke and Wulfram Gerstner. Hebbian plasticity requires compensatory processes on multiple timescales. Philosophical transactions the Royal Society of London. Series B, Biological sciences, 372(1715), 2017. doi: 10.1099/rstb22016.222.
[53] Dennis L H Kruijssen and Corette J Wierenga. Single synapse LTP: A matter of context? Frontiers in cellular neuroscience, 13:496, 2019. doi: (0.3389/fncel.2019.0000 9.0099..
[54] Li Deng. The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 29(6):141-142, 2012.
[55] Han Xiao, Kashif Rasul, and Roland Vollgraf. fushion-mnist a novel image dataset for benchmarking machine learning algorithms, arXiv, 2017. doi: https://doi.org/1O.ar wwt..oo////...5///aiiiiiii. 1708.07747.
[56] Michael J Higley. Localized GABAergic inhibition of dendritic ca(2+) signalling. Nature reviews. Neuroscience, 1599):567-572. 2014. doi: 10. 10.10088mm3803.
[57] H Hasuo and T Akasu. Activation of inhibitory pathways suppresses the induction of long-term potentiation in neurons of the rat lateral septal nucleus. Neuroscience, 105(2):343-352, 2001. 2001. doi: 00000////0000-4520000000000
[58] R M Douglas, G V Goddard, and M Riives. Inhibitory modulation of long-term potentiation: evidence for a postsynaptic locus of control. Brain research, 240(2):259-272, 1982. doi:
00.00660006-89333229002211
[59] Vibhakar C Kotak, Ana Mirallave, Todd M Mowery, and Dan H. Sanes. GABAergic inhibition gates excitatory LTP in perirhinal cortex. Hippocampus, 27(12):1217-1223, 2017. doi: 10.
1002/hipo.2277 22799.
[60] Tatsuya Hayama, Jun Noguchi, Satoshi Watanabe, Noriko Takahashi, Akiko Hayashi- Graham R Ellis-Davies, Masanori Matsuzaki, and Haruo Kasai. GABA promotes the compet- itive selection of dendritic spines by controlling local ca2+ signaling. Nature neuroscience, 16 (10):1409-1416 2013. doi: 10.1038/nn.3499
[61] Vincent Paille, Elodie Fino, Kai Du, Teresa Morera-Herreras, Sylvie Perez, Jeanette Hellgren Kotaleski, and Laurent Venance. GABAergic circuits control ppike-timing-depeeee plasticity. The Journal of neuroscience, 33(22);9353-9366 2013. 2013.
doi: 10. N0.1523/NEUROSCII57766 12.
[62] Steffen B E Wolff, Jan Gründemann, Philip Tovote, Sabine Krabbe, Gilad A Jacobson, Christian Müller, Cyril Herry, Ingrid Ehrlich, Rainer W Friedrich, Johannes J Letzkus, and Andreas Lüthi. Amygdala interneuron subtypes control fear learning through disinhibition. Nature, 509(7501): 453-458, 2014. doi: 10. 10.100/natuue 13358..
[63] Sabine Krabbe, Enrica Paradiso, Simon d' Aquin, Yael Bitterman. Julien Courtin, Chun Xu, Keisuke Yonehara, Milica Markovic, Christian Müller, Tobias Eichlisberger, Jan Gründemann, Francesco Ferraguti, and Andreas Lüthi. Adaptive disinhibitory gating by VIP interneurons permits associative learning. Nature neuroscience, 22:11):1834-188 2019. doi: 10. 1038/ -41593001-00508y
14

---

[64] Flavio Donato, Santiago Belluco Rompani, and Pico Caroni. Parvalbumin-expine expresssin basket- cell network plasticity induced by experience regulates adult learning. Nature, 504(7479): 272-276, 2013. doi: 10. (038/nature12266
[65] Robert C Froemke, Michael M Merzenich, and Christoph E Schreiner. A synaptic memory trace for cortical receptive field plasticity. Nature, 450(7 168):425 429, 2007. doi: 10.1038/
nature06289.
[66] Johannes J Letzkus, Steffen B E Wolff, Elisabeth M M Meyer, Philipp Tovote, Julien Courtin, Cyril Herry, and Andreas Lüthi. A disinhibitory microcircuit for associative fear learning in the auditory cortex. Nature, 480(7377) 2011. doi: 10. 1038/nature 1067//
[67] Leena E Williams and Anthony Holtmatt. Higher-Order thalamocortical inputs gate synaptic Long-Term potentiation via disinhibition. Neuron, 101(():91-102.e) 2019. doi: 10.1016/j.
couron.2018. 10..049.
[68] Stéphane Pagès, Nicolas Chenouard, Ronan Chéreau, Vladimir Kouskoff, Frédéric Gambino, and Anthony Holtmaat. An increase in dendritic plateau potentials is associated with experience- dependent cortical map reorganization. Proceedings of the National Academy of Sciences of the United States of America, 118(9), 2021. doi: 10. (0.1073/pnas.2222922111
[69] Martha Canto-Bustos, F Kathryn Friason, Constanza Bassi, and Anne-Marie M Oswald. Dis- inhibitory circuitry gates associative synaptic plasticity in olfactory cortex. The Journal of neuroscience, 42(14):2942 2222292222990,, 2022. doi: 10. 1
[70] Johannes Letzkus, Steffen B Wolff, and Andreas Lüthi. Disinhibition, circuit mechanism for associative learning and memory. Neuron, 88(2):264-276, 2015. 2015.09.024.
doi: 10. 10.1016/j.ncuron
[71] Setsuko Sahara, Yuchio Yanagawa, Dennis D M Leary, and Charles F Stevens. The fraction of cortical GABAergic neurons is constant from near the start of cortical neurogenesis to adulthood. The Journal of neuroscience, 32(14))4755 4761, 2012. doi: 10.5233/NEUROSCC..622.11 2012.
[72] T.P Vogels, H Sprekeler, F Zenke, C Clopath, and W Gerstner. Inhibitory plasticity balances excitation and inhibition sensory pathways and memory networks. Science, 334(6062): 1569-1573, 2011. doi: 10.1 10.1126/science.. 121 1095.
[73] Guillaume Hennequin, Everton Agnes, and Tim Vogels. Inhibitory plasticity: Balance, control, and codependence. Anmual review of neuroscience, 40:557-579, 2017. doi: 10.1146/ --u-uv---euuo-02111---333
[74] Lang Wang and Arianna Maffei. Inhibitory plasticity dictates the sign of plasticity at excitatory synapses. The Journal of neuroscience, 34(4):1083-109) 2014. doi: 10. 10.152//NEUROSCI. 4711-13.2014.
[75] James D' D'amour and Robert Froemke. Inhibitory and excitatory spike-timing-dependenn plasticity in the auditory cortex. Newron, (6(2):5144522) 2015, 014.
doi: 00.10166..eeuron20111
[76] Jonathan Mapelli, Daniela Gandolfi, Antonietta Vilella, Michele Zoli, and Albertino Bigiani. Heterosynaptic GABAergic plasticity bidirectionally driven by the activity of pre- and postsy- naptic NMDA receptors. Proceedings of the National Academy of Sciences of the United States of America, 113(55)9898-99933, 2016. doi: 10. mt.1077/pnss.. 1601 194113.
[77] Everton J Agnes and Tim P Vogels. Codependent excitatory and inhibitory plasticity accounts for quick, stable and long-lasting memories in biological networks. bioRxiv, 2022. doi: 10.1101/2021.04 1.2...0.......... 03.9966...
[78] Christoph Michl and Julijana Gjorgjivaa. Stability and learning in excitatory synapses by nonlinear inhibitory plasticity. PLoS computational biology, 18(12):e101088) 2022. doi: 10.1371 10b.100062.
15

---

Appendix A
Comparison with explicitly error-modulated plasticity rules
Normative theories functional learning in neuronal circuits of the brain commonly derive biolog- ically plausible solutions for the credit assignment problem that approximate BP. Like BP, these models result in explicitly error-modulated update rules for synaptic weights of the general form
Awg x prej x (post); x C₁
(12)
where pre; is presynaptic neuronal activity, (post)), ; is a nonlinear function g(.) of postsynaptic neuronal activity and is neuron-specific error signal that determines the sign of plasticity. In this section, we showcase how error-modulated plasticity rules drive neuronal activity to a stable fixed point and discuss their predictions for in vitro electrophysiology experiments (Supplementary Fig. S1).
Dendritic error rules
b)
4,0
2M 3.0
pre
pre
e O 1
Ub
Firing rate #[a.u.]
Firing rate Fla.ul]
Firing rate Flau.
10
Deep Feedback Control
Burst-dependent plasticity
g)
Aow
2000
5
10
10
Firing rate [[a.u.]
Dend. voltage vala.u.]
Event Rate [a.u.]
Poisson rate [a.u.]
Figure S1: Simulated in vitro patch clamp experiments with different learning rules proposed for bio-plausible credit assignment. (a) Dendritic error rules [I] encode the eπrоr (red) in the difference between somatic and basal dendritic voltage. When top-down feedback nudges neuronal activity towards a target, this results in error-modulated learning with stable fixed point at n n' (b) In the absence of closed-loop feedback circuit, dendritic error rules postulate that the dendritic potential is canceled out by recurrent inhibition, which would result in weight changes limited to LTD. (c) When isolating single neuron without recurrent connections from excitatory to inhibitory neurons, no weight change can take place except by experimentally injecting currents into dendritic compartments. Increasing inhibitory activity experimentally should thus cause LTD. (d) DFC models [2, 3] use a dendritic error learning rule that compares dendritic (only ford-rrr to somatic (including feedback) activity. In the presence a top-down controller, this leads to error-modulated learning. (e) Similar (c), in the absence of top-down feedback, the sign of plasticity is determined by externally injected current into the somatic compartment.
(f) In models using burst-dependent plasticity [4. 5], the error (red) is encoded in the deviation of baseline apical membrane potential, which determines the burst probability of the neuronal output. In a closed-loop circuit, this leads to uuror-modllate learning. (g) If firing rates are Poisson distributed and thus burst probability increases as a function of firing rate, such models can resemble plasticity with postsynaptic threshold. this adapted model, increasing inhibitory currents into the apical compartment bias the learning rate towards LTD.
Dendritic error rules. Dendritic error rules employ neuron models with multiple segregated compartments to encode a neuron- -seurfn-peccficccccc eтror signal e; in the difference between neuronal activity of different compartments [1. 6]. Here, we implement the model described by Sacramento al. [1]. in which the error is encoded in the difference between somatic voltage Uy and basal dendritic voltage up. For simplicity, we consider a single pyramidal neuron receiving presynaptic input r weighted by feed-forward weight w. Top-down feedback provides a current to an apical dendritic compartment dA that is canceled out by recurrent inhibition, leaving an error signal the apical dendrite that in turn influences the somatic voltage (Supplementary Fig. Sla). The error signal used
16

---

for the but weight update is then decoded from the difference between the output of the neuron, r c(us). and the output that would be observed if the apical dendrite potential (i.e. the top-down error) was zero. Specifically, the weight change at synapse with presynaptic input is given by
Aw Dendr. Eiror
1f ( (us) Φb (un))= x
(13)
where ûG is a function of the dendritic potential that takes into account dendritic attenuation parameters of the model, <(.) is a nonlinear activation function and r) is a learning rate (see Sacramento et al. [1] for detailed description of the model dynamics).
When this microcireuit is intact, top-down feedback is proportional to a target firing rate yd and the neuron-specific error determining the sign of plasticity is proportional to the output error yor Consequently, the synaptic weight will increase when the neuronal firing rate below the target rate and vice versa (Supplementary Fig. S1a).
In contrast, when the apical dendrite does not receive top-down inputs, for example when top-down afferents are severed in an in vitro slice preparation, the plasticity rule in Eq. (13) cannot decode an error. In this setting, recurrent inhibition would still try to cancel out an expected top-down input to the apical compartment, leading to negative apical potential and in turn to LTD (Supplementary Fig. Slb). Note that in other implementations of dendritic error rules, that do not employ recurrent inhibitory circuit [6, 7], Aw = O in this setting.
If the local inhibitory microcircuit is also impaired, for example by blocking recurrent connections from excitatory neurons to inhibitory neurons, the apical voltage is independent of presynaptic stimulation. In this isolated neuron setting, which arguably resembles most experimental conditions for in vitro clectrophysiology. dendritic eπrоr rules would predict no weight change, except if current injected into the apical compartment experimentally, in which case the weight change would be proportional to the injected current (Supplementary Fig. S1c)
Deep Feedback Control. The DFC framework put forward by Meulemans et al. [2, 3] also makes use of a dendritic error rule, but considers simpler microcircuit without recurrent inhibition. The weight change in DFC is proportional to the difference between a neuron's 's bottom-up inputs, encoded in basal dendritic potential ug, and its somatic potential us. The somatic potential integrates both bottom-up inputs and a top-down control signal that moves the neuronal firing rate towards the target pl"r The DFC update rule is defined as
DFC
17 (us UB) x
(14)
= 17 (us - wx) x
(15)
When the top-down control loop is intact, the DFC learning rule resembles the learning dynamics of dendritic error rules and creates a stable fixed point at r p (Supplementary Fig. S1d). As was the case in dendritic error rules, when the top-down control signal is impaired, the sign and magnitude of the weight change are proportional to externally injected currents (Supplementary Fig. Sle).
Burst-dependent plasticity. Payeur et al. [4] suggested an alternative mechanism to leverage segregated neuronal compartments for error-driven plasticity. As in dendritic error rules, burst- dependent plasticity relies on top-down feedback connections an apical dendritic compartment. However, in contrast to the previously described learning rules, burst-dependent plasticity does not rely on compute e; using differences in neuronal activity between compartments. Instead, burst- dependent plasticity exploits the unique spiking behavior of cortical layer pyramidal neurons [8], in which inputs to the apical dendrite determine whether bottom-up inputs to the neuron result in a single output spike ("event") or several output spikes fired in quick succession ("burst"). This approach allows the neuron multiplex forward signals are encoded in the event rate revent while top-down errors are encoded in the burst probability p. To illustrate the dynamics arising from burst-dependent plasticity, consider the rate-based version of the model proposed by Greedy et al. [5], for which the weight change is defined as
streams of information in its firing rate: bottom-up
(16) where pa is a parameter denoting the baseline burst probability. The sign of plasticity is thus determined by deviations from the baseline burst probability (see Greedy et al. [5) for details on model dynamics).
Aw Burst-dependent 17 Tevent (y Ph)
17

---

In the intact, closed-loop microcircuit, top-down feedback communicates an error signal to the apical compartment that in- or decreases the burst probability depending on whether the neuronal output Yevent is smaller or larger than the target rate plar Similar to other error-modulated plasticity rules, this creates stable fixed point for plasticity at revent plar (Supplementary Fig. S1f).
Because the baseline burst probability ps acts like postsynaptic plasticity threshold. burst-dependent plasticity can resemble some aspects of phenomenological Hebbian plasticity rules. Specifically, when synaptic inputs to the neuron follow a Poisson distribution over time, its output will also be Poisson distributed and the burst probability is not solely determined by top-down apical inputs. To illustrate this scenario, we consider an isolated neuron without top-down feedback to its apical dendrite. Given diisso--ittibuuted inputs 2, we can assume that the burst probability of the postsynaptic neuron is determined by the input Poisson rate, i.c. p= fzz) Plotting the weight change as function of the input Poisson rate while keeping the baseline burst probability ps fixed results in learning rule that resembles phenomenological Hebbian plasticity with postsynaptic threshold (Supplementary Fig. SIg).
Appendix B Analysis of the learning rule for dis-inhibitory control
Let's first recall the learning rule expressed in Eq. (9), where local error is encoded in the difference between the excitatory firing rate and inhibitory membrane potential. To enhance clarity, we reformu- late the weight change for a singular synapse, Way, established between presynaptic excitatory neuron j and postsynaptic excitatory neuron
Awty o' (u,) E
(1.
ut) T..
(17)
where we assume all quantities are evaluated at equilibrium state. Here, rf. and r represent the pre- and post-synaptic firing rates respectively. of symbolizes the postsynaptic membrane potential, whereas if denotes the membrane potential of the inhibitory neuron linked to the postsynaptic cell. This learning rule is non-local since the postsynaptic neuron doesn't 't possess direct access to the membrane potential of an inhibitory neuron u!.
ne
To navigate around this limitation, introduced a linear approximation of the inverse inhibitory activation function in Eq, (11), yielding a local learning rule of the generic form:
Awiy = ø' (u (+ - 5r!) T
(18)
The simplified microcircuit model we studied here assumes a particular form of one-to-ons connec- tivity in which each excitatory neuron only receives input from one inhibitory neuron and vice-vers versa. Conceptually, rf can thus be interpreted as the inhibitory current received by the postsynaptic neuron.
B.1 Relationship between L and H
Following [3], it is straight forward show that reduction in H leads to a reduction in L by observing that, if H. is minimized to zero, L is also minimized to zero. This is simple to show by observing that H. = 0 only occurs when C 0 (cf. Eq. (4)), i.c when the uncontrolled network output equals the target. Formally, for any given loss that evaluates to zero rf F i.c. L (x,x) 0. we have
H- Σ (m) = 0 = Σ (EL
(TL (n), rT. true (n))
(19)
where ig denotes the output of the network in the absence of a controller. However, in contrast the networks considered in [3], the dis- dis-inhiby controller considered in this work is not guaranteed to minimize the surrogate loss H. and thus L to zero
B.2 Linear approximation of the inverse function
The learning dynamics described by the learning rule in Eq. (11) depend implicitly on the chosen nonlinear activation function r = d(u), which dictates the error induced by the linear approximation. For all simulations in this work, we use soft rectifying nonlinearity as neuronal activation function:
d(u)= log(1 + exp(u 7))
(20)
18

---

which is parameterized through its scale 3 and shift Additionally, the learning dynamics are contingent on the choice of linearization parameters 0 and used approximate the inverse activation function,
Φ (r) = 7 + log exp
-1) + dr.
(21)
Fig. S2 depicts functionally and phenomenologically different learning dynamics the same three settings already investigated in Section 4, that can be obtained by changing the linearization parameters.
Linear approximation activation function
Closed-oop feedback
Microcircuit feedback
Direct inhibition
Inverse Approx.
/M 3.0
15
1.5
10
Aiw
1.0
8- 1-5
0.5
0.5
2.0
1.5
bw
1.0
10
6 1.5
5
10
2
0
0
2
2.0
30
1.5
Ja.u.
Ai
1.0
10
0.5
0.5
0-
5
10
15
2
0
2
2.0
15
1.5
1.0
4.0 4.0
0.5
0 1.0
2.0
1.5
bw
1.0
10
0.5
0
5
10 e [a.u.]
15
0
2
0
5
10
0
2
Pla u.]
Fla
F(a.u ]
Figure S2: Effect of varying linearization parameters 0 and on the learning rule in different microcircuit conditions. The first column depicts the inverse activation function Eq. (21) and several linear approximations thereof, parameterized through the intercept 0 and slope The second column depicts the dynamics of the resulting learning rule in a microcireuit under the influence of a top-down disinhibitory controller. The third column depicts the learning dynamics in the absence of top-down control. The last column depicts the learning dynamics under simulated experimental conditions, with direct control over the inhibitory current.
19

---

For the training of the deep neural networks discussed in Section 5, we adopted practical approach and linearize the inverse function using a first-order Taylor expansion. This method effectively reduces the linear approximation to a single hyperparameter, the linearization point i. Therefore, we approximate the inverse activation function as follows:
x-((r) - f(r) ((F) + (((((r
(22)
where (r) denotes the derivative of the inverse activation function evaluated at the lincarization point r. From this linearization, we can extract the parameters 0 and ó in relation to F:
f() (f
8
(23)
Denoting ū - ( can the inverse function theorem formulate the relationship between the linearization point F and the general linearization parameters, and 5, in a more interpretable fashion:
0 ū
8
(24)
& ii)
Appendix C Simulation details
All numerical simulations were implemented using Python 3.8.10 and were executed on NVIDIA Quadro RTX 5000 GPUs. The software stack includes Jax [9], Flax [10], and Diffrax [11]. Datasets were obtained from the TensorFlow datasets library [12]. The code to reproduce our results is publicly
available at sttthhuh.oommmmm--aasllliiiiiiii
C.1 Single-synapse experiments
For the investigation of learning dynamics at single synapse (Section 4), we simulated a single microcircuit unit. The microcircuit unit was driven by increasing the strength of afferent input into the excitatory neuron. To capture the input-output curve of neurons in the absence of any background inputs, we parameterized the neuronal activation function Eq. (20) with 3 = 1.0 and y = 3.0 for both excitatory and inhibitory neurons. For each input strength, the microcircuit was simulated for 600 ms to ensure that neuronal dynamics settled to an equilibrium. Synaptic weight updates where computed the equilibrium state. Numerical integration was performed using the forward Euler method with a time step of ms.
C.2 Student-teacher task
Inputs. For the student-teacher learning task (Section 5), we generated 30 sine wave inputs with random frequency, amplitude and phase shift. Frequencies were randomly drawn from uniform distribution (0. 1Hz 2.0Hz). Similarly, sine amplitudes were randomly drawn from the distribution U(001, 1.0) and phase shifts were drawn from the distribution (1, T). Sine waves were continuously generated and used as input to both the student and teacher network during training. Evaluation for panel B in Fig. 4 before and after training was performed on a 5-second window of the training data.
Feedback weights. Feedback weights Q, were initialized proportional to the normalized network Jacobian the beginning of training:
(,
(0) = (O)]
(25)
where we normalized the Jacobian by dividing it by its Frobenius norm. The parameters a controls the norm of the effective Jacobian, and thus the magnitude of the feedback weights. We found that normalizing the Jacobian is not necessary for good training performance, but prevents exorbitantly large values in the feedback weights and can sometimes speed up numerical integration of the neuronal dynamics and aid the stability of the dynamical system.
As the forward weights W; change over the course of learning, the network Jacobian of the student network relevant for computing suitable feedback weights changes as well. To account for the lcarning-induce induced changes in the network Jacobian, make the feedback weights Q: part of the
20

---

TB Exc. membrane time constant network dynamics and continuously nudge them towards the normalized network Jacobian with slow time constant TQ:
dQ TQ di - +
Jt(t)
(26)
J((I
As a result, feedback weights are dynamically adjusted to retain good control performance. We chose TQ to be faster than the plasticity time constant Tw and much slower than changes in the input xtt). As a result, the feedback weights reflect the average network Jacobian over time, but adapt to changes in the neuronal dynamics caused by have demonstrated that feedback weights reflecting the average Jacobian can also be learned using a simple, fully local Hebbian learning rule based on noise correlations between neuronal activity and top-down control.
plasticity. Meulemans et al. [2, 3]
Training. Both student and teacher networks were randomly initialized using Kaiming initialization and hyperparameters relating to numerical integration and neuronal dynamies were kept identical for student and teacher networks (see Table S1). Student networks were trained continuously for a total of 5 minutes, split into 60 periods each lasting 5 seconds. For each 5-second window, we first computed the output of the teacher network to obtain target for the top-down controller. We then computed the output of the student network in an open-loop setting without top-down control to calculate the MSE loss L. Finally, and continuously monitored the least-control loss H. Numerical integration was performed using Tsitouras' 5th order explicit Runge Kutta method [13] with an adaptive step size.
simulated the student network with active top-down control
Table $1: Hyperparameters used for training networks on the student-teacher task (Fig. 4).
Parameter
Description
Value
20 ms
Be
Exc. activation function scale Exc. activation function shift Inh. membrane time constant Inh. activation function scale Inh. activation function shift Controller time constant Proportional control strength Integral control strength Weight update time constant Feedback weight update time constant
5.0
YE
3.0
5 ms
BE
1.0
YE
3.0
100 ms
kp
30.0
ki
15.0
TW
60 s
TQ
30
f
Linearization point for learning rule
0.5
Jacobian norm. constant
7.5
C.3 Computer vision benchmarks
Data preprocessing and network architecture. MNIST and Fashion-MNISTT images were rescaled to values between and and flattened before being used as inputs to the networks. Each hidden layer was composed of 256 excitatory-inhiiioo microcircuit units for networks trained with disinhibitory control or 256 neurons for networks trained with BP.
Classification with Softmax and cross-entropy loss. Conventionally, classification tasks are solved using a Softmax layer as output of the network. The Softmax layer is trained match a one-hot encoded label r As pointed out by Meulemans et al. [3], because the Softmax layer output only matches the one-hot vector when has an infinite input, this can lead to problems when feedback control is strong, Thus, we use soft targets where the value of r for is 0.99 for the correct target and 0.01/mg-1 - 1 for all other entries. Following [3), we use a linear output layer and absorb the Softmax operation into the cross-entropy loss function:
clussification
Σ rhe (n) log (Softmax (FZ(n)))
(27)
Our dis-inhibitory controller and the associated learning rules require nonlinear activation function and recurrent inhibition, which could interfere with the Softmax in the loss function. Thus,
21

---

Table S2: Hyperparameters used for training networks on computer vision benchmarks
Parameter
Description
Value
Exc. membrane time constant
20 ms
m
Inh. membrane time constant
5 ms
B
Activation function scale
1.0
Activation function shift Controller time constant
0.0
Tc
100 ms
kp
Proportional control strength
0.2
k;
Integral control strength
0.4
a
Jacobian norm. constant
1.0
implemented the readout layer for classification tasks linear layer without excitatory-inhiitory units:
TE LL = (r WorE-1() + QLe(1)
(28)
with QL I, and performed weight updates according the deep feedback control learning rule [3].
Hyperparameter optimization. Optimal values for the controller parameters kp and K were obtained using the Optuna package [14]. Optimization was performed using Tree-structured Parzen Estimator [15] with 25 starts. Specifically, hyperparameters were chosen to minimize the validation loss of 3-layer network after 20 training epochs using the exact inverse learning rule on the MNIST dataset.
Training. Network weights were initialized using Xavier initialization and trained for 50 epochs with batchsize of 100. For every input sample, we first let the network settle to an equilibrium in the absence of feedback control. This uncontrolled equilibrium state was used to calculate the feedback weights Q and used as initial state for the second convergence with top-down control. Numerical integration was performed using Tsitouras' 5th order explicit Runge-Kutte method [13] with an adaptive step size for maximum duration of seconds or until equilibrium was reached, defined by an absolute tolerance value of 10-6, Synaptic weight updates were performed at the controlled equilibrium state and averaged across each minibatch. For all experiments, we used the ADAM optimizer with with learning rate 0.001.
Feedback weights. Feedback weights were calculated proportional to the normalized network Jacobian:
J
Q
(29)
where J, is the Jacobian obtained at the uncontrolled equilibrium state (cf. Eq. (7)). Because this calculation results in nepe-dee feedback weights, which are biologically implausible, we also consider the case in which feedback weights reflect the average Jacobian. As demonstrated in Meulemans et al. [2. 3), such feedback connections could be learned locally making use of noise correlations between neuronal compartments receiving top-down feedback and the strength of the control signal. Here, we instead take a practical approach and take the average of the Jacobian across each minibatch:
J.
Avg
(Avg J)
(30)
|| A
j.
(31)
N n=l
22

---

Table S3: Validation accuracy in % for networks trained with BP or dis-inhibitory feedback control. Reported values are mean +/- stdev (n 10).
MNIST
Fashion-MNIST
hidden layer
hidden layers
hidden layer
hidden layers
BP Exact inverse
98.0 = 0.24
98.2 + 0.14
90.1 + 0.29
90.3 + 0.34
97.5 0.19
97.5 0.19
90.0 = 0.24
90.0 +\2 0.38
Linear threshold
96.9 + 0.16
96.4 + 0.18
88.8 + 0.25
88.1 + 0.25
Exact inverse (Avg J)
97.6 = 0.15
96.9 + 1.6
89.8 + 1.39
89.2 + 0.48
Linear threshold (Avg J)
96.4 = 0.15
95.8 + 0.21
87.9 + 0.36
86.2 + 0.52
23

---

References
[1] João Sacramento, Rui Ponte Costa, Yoshua Bengio, and Walter Senn. Dendritic cortical micro- circuits approximate the backpropagation algorithm. In S Bengio, H Wallach, H Larochelle, Grauman, N Cesa- Bisa-Bianchi, and Garnett, editors, Advances in Neural Information Process- Curran Associates, Inc., 2018.
ing Systems 31, pages
[2] Alexander Meulemans, Matilde Tristany Farinha, Javier Garcia Ordonez, Pau Vil- imelis Aceituno, João Sacramento, and Benjamin F. Grewe. Credit assignment in neural networks through deep feedback control. In M. Ranzato, A. Beygelzimer, Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 4674 4687. Curran Associates, Inc., 2021.
[3] Alexander Meulemans, Matilde Tristany Farinha, Maria Cervera, João Sacramento, and Benjamin F. Grewe. Minimizing control for credit assignment with strong feedback. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors, Proceedings of the 39th International Conference Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 15458-155 1483., PMLR, 2022.
[4] Alexandre Payeur, Jordan Guerguiev, Friedemann Zenke, Blake A Richards, and Richard Naud. Burst-dependent synaptic plasticity coordinate learning in hierarchical circuits. Nature neuroscience, 2021. doi: 10. 1038//41599-002-00885777
[5] Will Greedy, Heng Wei Zhu, Joseph Pemberton, Jack Mellor, and Rui Ponte Costa. Single-phase deep learning cortico-cortical networks. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 24213 24225. Curran Associates, Inc., 2022.
[6] Robert Urbanezik and Walter Senn. Learning by the dendritic prediction of somatic spiking. Neuron, 81(3):521-528, 2014. doi: 10.10166f..nuuree new.oo..nnnroo11....3. 11.03..
[7] Jordan Guerguiev, Timothy P Lillicrap, and Blake A Richards. Towards deep learning with segregated dendrites. eLife, 6, 2017. doi: 10. 10.7554/CLife.22900
[8] M E Larkum, J Zhu, and B Sakmann. A new cellular mechanism for coupling inputs arriving different cortical layers. Nature, 398(6555) 1999. doi: 10.1038/1 8686.
[9] James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye MindermmnnMinnee and Qiao Zhang. JAX: composable transformations of Python+NumPP programs, 2018. URL http:// //gthuub com/ coom/gogge//axx
[10] Jonathan Heek, Anselm Levskaya, Avital Oliver, Marvin Ritter, Bertrand Rondepierre, Andreas Steiner, and Mare van Zee. Flax: neural network library and ecosystem for JAX, 2023. URL http://githin com com/gooole//fl.
[11] Patrick Kidger. On Neural Differential Equations. PhD thesis, University of Oxford, 2021.
[12] TensorFlow Datasets, collection of ready-to-use datasets. https: https::/awx.ttensorllooom org/ datasets.
[13] Ch Tsitouras. Hange-Kuttt pairs of order 5(4) satisfying only the first column simplifying assumption. Computers mathematics with applications, 62(2):770-775, 2011. doi: 10.1016/j. camwa.201 .06.002.
[14] Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. Optuna: next-generation hyperparameter optimization framework. arXiv, 2019.
[15] James Bergstra, Rémi Bardenet, Yoshua Bengio, and Balázs Kégl. Algorithms for Hyper- Parameter optimization. In J Shawe-Taalor,, R Zemel, Bartlett, F Pereira, and K Q Weinberger, editors, Advances in Neural Information Processing Systems, volume 24. Curran Associates, Inc., 2011.
24


Reviews:
[{"review_id": "TO6r9TU01c", "reviewer": "Reviewer_z6yQ", "summary": "This paper adds recurrent inhibition to each layer of a DNN architecture in order to facilitate a more biologically-plausible form of credit assignment. It shows how this circuit can explain some of the features of plasticity found in vitro and that DNNs with this circuit can learn to perform simple visual tasks. ", "questions": "Why is there no weight matrix for the recurrent connections?\n\nCan the authors elaborate on how their formulation supports stability? For example, I didn't fully understand this claim \"On the other hand, our model suggests that a rapid compensatory mechanism could be implemented as a combination of recurrent inhibitory microcircuits and a linear inhibitory threshold in the synaptic plasticity rule\"\n\n\"Experiments on excitatory plasticity in vitro are commonly performed in the presence of Tetrodotoxin, a sodium channel blocker, to minimize interference of inhibitory activity\" Doesn't TTX block all activity, not just inhibitory?\n\n\"Specifically, we block recurrent inhibition \" From the diagram it looks like you block recurrent excitation, not inhibition (There would be no point in varying the activity of the inhibitory neuron if its outward connections were blocked). \n\nIs \"in the absence of any inhibitory input\" supposed to say \"absence of input to the inhibitory cells\" ?\n\nIt's not clear to me why the \"postsynaptic\" term in equation 9 drops out when the E->I connection is dropped\n\n\"we manually set the feedback weights to the transposed network Jacobian at each time step\" does this mean that the feedback is different at each timestep? That is not very biologically-plausible.  ", "limitations": "Already mentioned above", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The microcircuit is biologically motivated\n\nThe insights provided into how the artificial experimental conditions in plasticity studies lead to specific results is helpful\n\n", "weaknesses": "The motivation and innovation was not entirely clear to me. The background is focused on how credit assignment requires distant neurons to interact, yet the problem of how information reaches each layer in the network is not what is tackled here. There is also discussion of the microcircuit \"decoding\" the credit signal, but it seems like the credit signal is fairly directly given to the layer, and so the microcircuit is more of a relay than a decoder. \n\nOne of the main results seems to be that a linear approximation to the inverse activation function can make the weight update rule slightly more biologically realistic and still works fairly well. This is fine, though not a very impactful result. \n\nI found some of the results descriptions confusing (see below)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> The motivation and innovation was not entirely clear to me. (...)\n\nWe realize that we did not explain the paper's motivation clearly. What our article contributes is a plausible mechanism consistent with experiments of how neuronal circuits can translate credit signals into weight changes. We do not propose a new strategy on how these credit signals are propagated. Thus our article is more about how feedback afferents can control the sign of plasticity at individual neurons. \n\nWe will rewrite the relevant parts in the introduction and discussion sections and propose a new title: “Dis-inhibitory neuronal circuits can control the sign of plasticity”, inspired by Richards & Lillicrap (2018).\n\n> (...) it seems like the credit signal is fairly directly given to the layer, and so the microcircuit is more of a relay than a decoder.\n\nYes and no. We agree that the feedback is given directly to the layer. But for a neuron to use this feedback for plasticity it needs to estimate the error signal from the magnitude of inhibitory current. Specifically, it needs to correct for its own contribution to the received recurrent inhibition. In that sense, we do think the term “decode” for both the learning rule and the circuit is justified. However, we will work this point out more clearly in our Discussion.\n\n>(...) a linear approximation to the inverse activation function can make the weight update rule slightly more biologically realistic and still works fairly well. This is fine, though not a very impactful result.\n\nWe respectfully disagree with this assessment. We think it is impactful because it provides a missing piece of the puzzle of how existing works on credit assignment could plausibly translate credit signals into local weight changes. Most previous models did not link their work to phenomenological learning rules observed in neurobiology. \n\n>Why is there no weight matrix for the recurrent connections?\n\nOur model adopts a one-to-one connectivity structure between excitatory and inhibitory neurons. Although biologically implausible, this minimal architecture is sufficient to demonstrate the central notion of our article: credit signals could be relayed through dis-inhibitory afferents, encoded within inhibitory currents and subsequently decoded using Hebbian plasticity mechanisms.\n\nOther prominent models, such as Sacramento et al. (2018), have employed similar 1-1 connectivity paradigms, underlining their utility in specific contexts. While we have some promising preliminary results on more realistic E-I connectivity patterns (see Section 3 of the general rebuttal and Fig. 2 of the attached PDF), a detailed study of this topic goes beyond the scope of this article. \n\n>Can the authors elaborate on how their formulation supports stability? (...)\"\n\nIn traditional Hebbian plasticity rules, runaway LTP can lead to unstable dynamics. As highlighted by theoretical studies (e.g, Zenke & Gerstner 2017, Yger & Gilson 2015), plasticity should be accompanied by a rapid compensatory mechanism, i.e. the learning curve should dip down into the LTD region at high firing rates or synaptic weights, to counterbalance a positive feedback loop. However, there is inconclusive evidence for such a mechanism from pair-based-recordings. Interestingly, the model presented in this article shows the desired behavior, but it arises as a circuit phenomenon (See Figure 2a-b in the Manuscript) from an interplay between two features:\n- Recurrent Connectivity: Increases in excitatory firing rate lead to increases in inhibitory current due to recurrent connectivity.\n- Linear error decoding: Without a top-down error signal to decode, the linear model decoding the error from inhibitory activity can overestimate the control signal, causing LTD. The linearization parameters play a key role in the stability of the learning rule, as illustrated in Fig. 4 of the Supplementary Material.\n\n>\"Experiments (...) are commonly performed in the presence of Tetrodotoxin (...) to minimize interference of inhibitory activity\" Doesn't TTX block all activity, not just inhibitory?\n\nThank you for pointing this out. TTX is commonly used in glutamate uncaging experiments to reduce interference from background activity. In paired patch-clamp recordings, experimenters instead often apply GABA antagonists, which specifically block inhibitory currents, or discard recordings in which background activity was present, in order to isolate the synapse under investigation.\n\nWe will change the sentence to:\n“Experiments on excitatory plasticity are commonly performed under conditions designed to minimize the interference of inhibitory activity, for example by applying GABA antagonists.\n\n>(...) From the diagram it looks like you block recurrent excitation, not inhibition (...).\n\nWe will change the sentence to:\n\"(...) specifically, we block recurrent connections from excitatory to inhibitory neurons, so that inhibitory activity is independent of excitatory activity (...)\"\n\n>Is \"in the absence of any inhibitory input\" supposed to say \"absence of input to the inhibitory cells\" ?\n\nWe will replace this phrase with “In the absence of any inhibitory activity, i.e. $r^I = 0$, (...) ”.\n\n>It's not clear to me why the \"postsynaptic\" term in equation 9 drops out when the E->I connection is dropped\n\nThank you for spotting this error. When the inhibitory firing rate equals zero, the learning rule reduces to $\\Delta w \\propto r_{\\text{pre}} \\left(r_{\\text{post}} - \\theta \\right) \\phi’(r_{\\text{post}})$.\n\n>\"we manually set the feedback weights to the transposed network Jacobian at each time step\" does this mean that the feedback is different at each timestep? (...).\n\nThanks for raising this point. Indeed, in our initial experiments, the feedback weights were dynamically adapted at each timestep based on the transposed network Jacobian. We have conducted additional simulations using static feedback weights (see Section 1 in the general rebuttal and Tables 1&2 in the attached PDF)."}, {"review_id": "L64OHYh0Oy", "reviewer": "Reviewer_EPnb", "summary": "This paper uses adaptive control theory to derive plasticity rules for a fairly plasubile model of multi-layer networks in the brain, which are capable of matching the performance of backpropagation without restrictive assumptions such as mirrored or very weak feedback connections. Specifically, each excitatory neuron has a dedicated inhibitory neuron, which is in turn inhibited by the error signal propagated along feedback connections. The plasticity rule for excitatory units is essentially Hebbian but modulated by the inhibitory signal, matching in vitro plasticity experiments. A brief theoretical derivation of the rule is augmented with experimental results on both toy and relevant problems. ", "questions": "* Line 150: learning is duplicated\n* Figure 2(a): The inset mentioned in the caption seems to be missing", "limitations": "All of the concerns I had were addressed thoroughly in the discussion section.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "* This work presents a relatively well-founded model for feedback of error in the brain, which seems to me one of the most biologically-plausible models of backprop to date.\n* All of its presentation is clear and should be accessible to readers with a wide variety of related backgrounds.", "weaknesses": "* There are some lingering issues of biological plausibility: There are still significant constraints placed on the feedback weights; each excitatory neuron is assumed to have its own inhibitory neuron (which doesn't match the ratios found in the brain); and Dale's law is not enforced.\n* The experimental evaluation is a bit terse; I would like to see the performance of this learning rule on a broader range of tasks (especially a non-toy problem with some temporal structure).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ">There are still significant constraints placed on the feedback weights\n\nWe acknowledge that there are constraints on the feedback weight matrix. Specifically, we used the transpose of the network Jacobian to obtain the feedback weights, which caused feedback weights to vary over time. To address these constraints, we have repeated our simulations with feedback weights derived from a stationary Jacobian. In fact, this has improved our results. Please refer to Point 1 in the general rebuttal and the attached PDF for a thorough discussion of these simulations. \n\n> each excitatory neuron is assumed to have its own inhibitory neuron (which doesn't match the ratios found in the brain)\n\nThe proposed model indeed relies on (1) biologically unrealistic one-to-one connectivity and (2) ratios between excitatory and inhibitory neurons. Reducing the number of interneurons to reflect neurobiological ratios has complex implications for the dimensionality of the credit signal, which to address is beyond the scope of this paper. However, we have some exciting preliminary data on low-rank feedback signals (see section 3 in our general rebuttal and Figure 2 in the attached PDF) that suggests that these issues can be overcome. We intend to address this question in the future. \n\n>Dale's law is not enforced\n\nDale’s law is enforced within each layer, where the E-I connections and I-E connections are purely excitatory or inhibitory, respectively. Output connections from one layer to the next are indeed not Dalian and can be both negative and positive. Such between-layer negative connections should be interpreted as being mediated through feed-forward inhibition. \n\n>The experimental evaluation is a bit terse; I would like to see the performance of this learning rule on a broader range of tasks (especially a non-toy problem with some temporal structure).\n\nThank you for this suggestion. We were not able to perform simulations on additional datasets due to their computational demand yet. We will try to train networks trained with our learning rule on several datasets with temporal structure, such as the Google Speech Commands dataset for keyword spotting and the Heidelberg Digits or TIDIGITS datasets for classification.\n\n> Line 150: learning is duplicated.\n\n> Figure 2(a): The inset mentioned in the caption seems to be missing\n\nThank you for spotting these typos. We will fix the mistakes in the revised manuscript.\n\n"}, {"review_id": "QGfVsqw5uU", "reviewer": "Reviewer_A5Rt", "summary": "The authors propose a neural plasticity mechanism with a key role for disinhibition. In particular, they apply a deep feedback control framework whereby feedback driven inhibitory neurons mediate changes in the feedforward excitatory connections. The proposed rule is argued to hold desirable properties in that it is local, captures/predicts experimentally observed plasticity, and can guide error-modulated learning. ", "questions": "- Fig 1a: what's the dfiference between error and feedback signals?\n- Fig 1b: what is f and g? Why are they different?\n- Fig 1 caption: eqprop not defined\n- line 93: \"While these error-modulated learning rules prove functionally useful...they fall short of capturing established properties of\nexperimentally observed plasticity, such as a postsynaptic activity threshold\". Forgive me if I'm being naive: what is meant here postsynaptic activity threshold? It's not clear to me.\n- line 99: \"However, the model (burstprop) assumes a rigid circuit architecture to decode errors from multiplexed spike trains and thus does not generalize to other neuronal circuits\". I don't understand the logic othis sentence: could the authors elaborate?\n- line 102: \"..necessitate feedback signals to be weak..\"; this sentence confused me a bit. It seems to me there's a difference between 1. the strength of feedback signals and 2. whether feedback signals activity influence neuronal activity. For example, in burstprop one might have a very high apical potential (and burst rate) but little/no influence on the event rate\n- line 130: define L\n- I found the explanation of how optimal feedback weights are found confusing (equation 7). Firstly, J contains L vectors for each u_i, but this is multiplied by only one u_i? Secondly, it seems equation 7 just shows how the change in r relates to any changes in u - this is true in whatever weights are chosen, I don't understand how this is an equation to be 'solved'. Finally, from what I understand in [1] it's true that that if the column space of Q is equal to the row space of J (line 145) then equivalence to Gauss-Newton optimisation is possible, but is this choice of Q necessarily optimal in this case?\n- equation 8: use of subindices meaning datapoints here whilst layers elsewhere is confusing\n- lines 152-159: it was only clear to me after looking at [2] that minimising the surrogate loss should also minimise the task loss. This should be presented more clearly\n- line 156: I don't see the logic of 'as a result' from the previous sentence; doesn't it just follow from equation 5?\n- line 167, 169: Eq. not equation\n- Fig 2: is the x axis post-synaptic rates? this should be in caption. Same with frequency in 2d\n- Fig 2b: inset diagram is unclear; is the black line the true inverse function? I am also curious to see an imperfect approximation where the approximation is above/below the true values at low/high values respectively\n- Fig 2 caption: 'resembles experimentally observed plasticity in simulated in vitro condition' - this seems a strong statement when only one paradigm (an isolated neuron) is actually shown\n- line 197: \"The dis-inhibitory feedback signal creates a stable equilibrium state for the weight dynamics that coincides with the postsynaptic target computed by the feedback controller\". I'm sorry, I didn't understand this sentence\n- for the student-teacher task in section 5, what is f? is it the output of a randomly intialised teacher network of the same architecture as the student network but without feedback? If without feedback, why does the solution for H in 3c bottom not go to zero? Also, is it necessary to set the feedback weights as the transposed Jacobian. Would it also work if its column space was equal to the row space of J?\n- For simulations in image tasks, n=3 seeds seems quite low to me. Could you repeat for a higher number? (like n=10)\n- For readability I'd recommend rotating table 2 (perhaps switching rows with columns is necessary)\n\ntypos: \nline 105. Full stop after references.\nequation 8: bold Q\nline 203: appendix section 'xyz' (also correct in appendix itself) \nline 207: poor grammar\nline 244: appendix C and D\nTabel 1 caption: full stop at end\nline 284: comma after 'fashion-MNIST'\nline 336: has -> have\n\nReferences: \nsee refs for weaknesses above", "limitations": "Limitations were well addressed by authors", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- the paper is generally well written (though there are some typos, see below)\n- the proposed plasticity is novel and well theoretically groundeded \n- the experimental predictions are well presented and appear feasible to perform\n", "weaknesses": "- I am not fully convinced of the extent of novelty within this work. It seems to me that the authors made only slight alterations to the setup in [1, 2] such that inhibitory neurons are now included in the architecture (instead of + Qc we now have - (- Qc)) . Of course the brain does include inhibitory neurons, so adding them explicitly is arguably one step closer to plausibility, but I think the authors could present a stronger argument by addressing more the functional/computational differences/benefits with this addition compared to the previous models [1,2]. Comparing it to theorem 2 in [1], is the key difference that the interneurons enable the network to avoid the need to store/discriminate between the feedforward (ff) activity and the total activity of the excitatory neurons? If so I think this could be more clearly conveyed\n\n- I think the authors could more in relating their work to other computational works which consider the role of inibitory neurons on plasticity. For example, would [3] make different predictions in Fig 2? Moreover, the authors do not relate their model to [4], which to my knowledge seems to have significant intersection in terms of modeling and possibly predictions. \n\n- though I found the writing in general good, there were still a fair few sloppy errors and places which were unclear to me (see below)\n\nReferences:\n[1] Meulemans et al. 2021, Credit assignment in neural networks through deep feedback control\n[2] Meulemans et al. 2022, Minimizing Control for Credit Assignment with Strong Feedback\n[3] Sacramento et al. 2018, Dendritic cortical microcircuits approximate the backpropagation algorithm\n[4] Greedy et al. 2022, Single-phase deep learning in cortico-cortical networks", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ">I am not fully convinced of the extent of novelty within this work. (...) is the key difference that the interneurons enable the network to avoid the need to discriminate between the feedforward (ff) activity and the total activity (...)\n\nWe realize that we did not sufficiently explain the advantage of a dis-inhibitory feedback circuit. The core *computational* advantage is that encoding the error in inhibitory activity removes the need for a neuron to store a copy of its feed-forward inputs (e.g. Sacramento et al. (2018), Meulemans et al. (2021, 2022)). \n\nOur model also improves upon existing work *conceptually* by demonstrating that a learning rule resembling phenomenological models of Hebbian plasticity can be used to decode the error and thus allow feedback connections to steer the sign of plasticity. This builds a plausible bridge between prior studies on credit assignment and the translation of credit signals into tangible local weight adjustments. \n\n>I think the authors could more in relating their work to other computational works  (...).\n\nWe agree that the comparison to existing models of bio-plausible credit assignment was not sufficient. We have repeated the simulated *in vitro* slice experiments for several related models (see Section 2 of our general rebuttal and Fig. 1 in the attached PDF)\n\n>Fig 1a: what's the dfiference between error and feedback signals?\n\nWe interpreted error signals as explicit relay of an error while feedback signals influence neuronal activity. \n\n>Fig 1b: what is f and g? Why are they different?\n\nWe mean two nonlinear functions of the postsynaptic activity. Phenomenological models usually assume a calcium trace. Models of credit assignment usually use the derivative of postsynaptic activity.\n\n>line 93: (...) what is meant here postsynaptic activity threshold?\n\nExperiments on Hebbian plasticity have found an activity threshold above which long-term depression changes to potentiation for co-activated synapses. In phenomenological models, this threshold is usually expressed in terms of a postsynaptic quantity (calcium / voltage / rate).\n\nFor example, Artola et al. (1990) found that the sign of plasticity was determined by postsynaptic voltage. Sjöström et al. (2011) observed that the sign of plasticity in an STDP protocol depended on postsynaptic firing rate. Lim et al. (2015) found evidence for a postsynaptic activity threshold by investigating *in vivo* firing rate distributions.\n\n>line 99: (...) I don't understand the logic othis sentence (...)\n\nWe realize this sentence was not clear. What we meant is that burstprop assumes electrically segregated dendritic trees that are limited to L5 pyramidal cortical neurons. It is unclear whether a similar mechanism would work for L2/3 neurons or other brain areas that do not show such a segregation.\n\n>line 102: (...) It seems to me there's a difference between 1. the strength of feedback signals and 2. whether feedback signals activity influence neuronal activity (...)\n\nThanks for raising this point. We will restructure our introduction section taking into account this categorization.\n\n>I found the explanation of how optimal feedback weights are found confusing (equation 7). (...)\n\nThank you for pointing out the lack of clarity. Given the changes to the feedback weights we use (see general rebuttal Section 1), we will update the whole paragraph on the choice of feedback weights $Q$.\n\n>(...) if the column space of Q is equal to the row space of J (line 145) then equivalence to Gauss-Newton optimisation is possible, but is this choice of Q necessarily optimal in this case?\n\nIndeed, the $Q$ we use is just one of many feedback weight matrices fulfilling this condition. In the final manuscript, we'll avoid referring to feedback weights as “optimal” to prevent misinterpretation. \n\n>Fig 2: is the x axis post-synaptic rates? (...)\n\nYes. We will make this clear in the figure caption.\n\n>Fig 2b: (...) is the black line the true inverse function? \n\nYes. We will edit the figure caption.\n\n>I am also curious to see an imperfect approximation where the approximation is above/below the true values at low/high values respectively\n\nThe impact of imperfect approximations is demonstrated in Figure 4 of the Supplementary Material in our original submission. We will make sure to highlight this supplementary Figure more prominently.\n\n>Fig 2 caption: 'resembles experimentally observed plasticity in simulated in vitro condition' - this seems a strong statement (...)\n\nWe will revise the sentence to:\n“(...) resembles plasticity observed in single-neuron electrophysiology experiments”\n\n>line 197: (...). I'm sorry, I didn't understand this sentence\n\nApologies for the confusion. We mean that the top-down controller influences the learning rule so that a stable fixed point exists when the neuron’s output equals the target. \n\n>for the student-teacher task in section 5, what is f? (...)\n\n$f$ is the output of a randomly initialized teacher network with a smaller hidden layer compared to the student. The teacher network is a conventional ANN without inhibitory units or feedback. One possibility the network does not reach exactly 0 error is the time-delay of the control signal in the online learning paradigm.\n\n>Also, is it necessary to set the feedback weights as the transposed Jacobian? (...)\n\nWe have reproduced our results using more relaxed assumptions on the feedback weights (see Section 1 of the general rebuttal and Tables 1-2 in the attached PDF). \n\n### Other suggestions\n\nThank you for many more helpful suggestions and finding several typos. We will address all typos and implement the following changes in the revised manuscript:\n\n- Define EqProp in Figure 1 caption\n- (line 156) change the phrase to “Thus, following Eq. (5)...”\n- (line 130) define $\\mathcal{L}$ \n- repeat all simulations n=10 times\n- rotate table 2\n- change the sub-indices in Eq. (8)\n- (lines 152-159) add a section on the relationship between $\\mathcal{H}$ and $\\mathcal{L}$ \n"}, {"review_id": "OBEJh8vAcp", "reviewer": "Reviewer_TQSB", "summary": "The authors extend previous work on deep feedback control (DFC) learning to make the functional form of plasticity a more realistic reflection of what is observed biologically. In particular the authors:\n1. Introduce a feedback control mechanism using targeted, neuron-specific inhibition signal that allows for synaptic plasticity that more closely resembles experimentally observed plasticity (e.g. BCM), compared to the error-based learning employed in a wide variety of previous papers.\n2. Show that learning with the extended plasticity rule still comes relatively close to backpropagation-level performance on several benchmarks (MNIST, FMNIST).\n3. Discuss at length the testable predictions of their extended model.", "questions": "Could you elaborate on the relationship between this algorithm and the algorithm proposed in Payeur et. al 2021? In particular, that algorithm also replicates BCM-like LTD/LTP phenomena--it seems as though the ability to replicate BCM-like learning dynamics is not unique to the DFC family of models. What are the key differences in testable predictions between this model and Payeur 2021? Or older predictive coding-based models like Urbanczik & Senn 2014?\n\nIs the method employed in this study inherently unique to the DFC family of algorithms, or does it apply equally well to the other algorithms as well? E.g., could similar interneuron-modulated plasticity also be applied to predictive coding-based models?\n\nIf the network is driven to a near-optimal performance regime by the controller for every stimulus, and this is occurring in a biological system, would there ever be any observable improvements in performance? Or would the animal be performing instantaneously very well, with the only observable progressive change being a reduction in the energy required for the controller? If this is true, which biological systems could this algorithm adequately model?\n\nIn this model, is top-down feedback exclusively isolated to inhibitory neurons? Is this compatible or incompatible with models that propose feedback is also (or exclusively) directed to apical dendrites of pyramidal neurons?", "limitations": "There are no obvious negative societal impacts of this work, and the authors very adequately address the limitations of their work.", "rating": 6, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 2, "strengths": "The strengths of the paper are as follows:\n1. Reducing an algorithm derived from optimization principles to the point that it can replicate LTD/LTP experiments is quite difficult, and many (but not all) preceding algorithms do not appear to succeed in this regard, relying more on error-based signals that are less clearly related to experimentally observed plasticity phenomena.\n2. Even with biophysically motivated modifications, the algorithm still performs quite well on image classification tasks, which is also difficult.\n3. The paper is very clearly written and organized.", "weaknesses": "The authors themselves identify several key weaknesses, which I will elaborate on below. However, to me the principal weakness is that the contribution is very incremental: previous studies (e.g. Payeur et al. 2021) have demonstrated that related learning algorithms can, in certain regimes, resemble BCM-like learning, and the high performance and 'locality' of the DFC family of algorithms has already been explored extensively in previous studies (e.g. Meulemans et al. 2021 & 2022). Therefore, it seems to me that the key improvement this study demonstrates is that the DFC algorithms can, with some extra tweaks, also resemble this type of Hebbian learning.\n\nOther weaknesses:\n1. As the authors note, their current learning scheme involves a controller with access to a highly complex Jacobian. This Jacobian is a function of the fixed-point of the network dynamics, and so as far as I can tell, for every individual stimulus, the network's feedback weights would have to be different in order to match the principled feedback control dynamics. Previous DFC studies demonstrated that it's possible to get away with simpler controllers with fixed weights, but this approximation is not used in this study, and it's not clear why--as is, though the learning rule locally resembles BCM, the feedback signal actually used to achieve high performance on MNIST/FMNIST is essentially almost as complicated than the backpropagation error signal itself.\n2. The unrealistic 1-1 mapping between inhibitory and excitatory neurons makes it difficult to pin down where exactly the controller feedback should be expected to be at the level of a cortical microcircuit.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> (...) to me the principal weakness is that the contribution is very incremental: (...)\n\nWhile our results are comparable to previous work in terms of performance, we argue that our model offers distinct computational and conceptual advantages:\n\nFirst, the dis-inhibitory controller we use causes the error signal to be encoded in the inhibitory current received by each neuron. This alleviates the need for the neuron to distinguish between its feed-forward inputs and overall output rate.\n\nSecond, our work demonstrates that a Hebbian learning rule with a minor tweak can decode top-down dis-inhibitory error signal and thus allow top-down feedback to steer the sign of plasticity. This learning rule resembles phenomenological plasticity models and thus provides a plausible mechanism through which credit signals can influence local plasticity rules. Thus, our work proposes a concrete circuit-level mechanism for error coding through modulation of recurrent inhibition.\n\n>(... this model) involves a controller with access to a highly complex Jacobian. (...) for every individual stimulus, the network's feedback weights would have to be different (...)\n\nWe have performed additional simulations in which the feedback weights are proportional to a stationary Jacobian. This actually improved the stability of our model and shows that our model’s performance is robust to more plausible feedback weights. (cf. Section 1 of the general rebuttal and PDF Tables 1-2 for details).\n\n>The unrealistic 1-1 mapping between inhibitory and excitatory neurons makes it difficult to pin down where exactly the controller feedback should be expected (...)\n\nWe agree that the one-to-one connectivity in our model is unrealistic. However, this minimal model is sufficient to illustrate the mechanism of dis-inhibitory control. It is yet unclear where top-down control of cortical circuits originates. In any case, we believe that dis-inhibitory interneurons (such as VIP+ cells) could be directly targeted by a controller. Alternatively, top-down feedback could consist of balanced excitatory and inhibitory currents. In this case, VIP+ interneurons could act as a gating mechanism for feedback control. \n\n>Could you elaborate on the relationship between this algorithm and the algorithm proposed in Payeur et. al 2021? (...)\n\nIn Payeur et al. (2021), the sign of plasticity is controlled by deviations from a baseline burst probability. Their proposed mechanism is plausible for cortical L5 neurons with a separate apical dendritic compartment spatially segregating top-down and bottom-up inputs. It is not clear how a burst-dependent mechanism would work for cortical L2/3 neurons or other brain areas that lack segregated dendritic compartments and do not show stereotypical burst firing.\n\n>What are the key differences in testable predictions between this model and Payeur 2021? Or (...) Urbanczik & Senn 2014?\n\nThe main difference between these three models is which quantity determines the sign of plasticity. Urbanczik & Senn (2014) postulates that the sign of plasticity depends on differences between local dendritic and somatic voltage. Burstprop (Payeur et al. (2021)) postulates that the sign of plasticity depends on the proportion of bursts in the output of the neuron. Our paper postulates that the amount of inhibitory current into the neuron determines the sign of plasticity.\n\n>(...) does (inhibition-modulated plasticity) apply equally well to the other algorithms as well? (...)\n\nYes! Thanks for asking. Although we derived our model within an adaptive control theory framework with a strong controller (strong nudge), the concept of decoding error signals from changes in inhibitory activity equally applies to models with weak feedback (weak nudge). For weakly nudged networks, our model is directly related to predictive coding models which are usually considered in the weak nudge limit (but see also Song, [...], Bogacz (2023)). \n\n>If the network is driven to a near-optimal performance (...) would there ever be any observable improvements in performance? (...) which biological systems could this algorithm adequately model?\n\nNo, we do not expect the animal to perform instantaneously very well. Feedback signals can only be computed when the animal received the feedback for its actions, e.g. a reward. \n\nFor simplicity, we considered the idealized regime in which an agent is presented with a completely unambiguous target signal immediately. A behavioral paradigm that comes close to this setting is fear conditioning, where a painful unconditioned stimulus (US) is a strong, unambiguous value target. Learning is expressed as a freezing response to the initially neutral conditioned stimulus (CS). If a control mechanism exists that drives a network initially responsive to the CS to respond to a target value given by the US, neurons that are part of the top-down controller should reduce their response to the US proportionally to the learned response to the CS. This has been observed for VIP+ interneurons in the amygdala (Krabbe et al. (2019)), corroborating a role for dis-inhibitory circuits in learning. \n\n>(...) is top-down feedback exclusively isolated to inhibitory neurons? Is this compatible (...) with models that propose feedback (...) to apical dendrites of pyramidal neurons?\n\nIn our model, feedback is exclusive to a specific class of inhibitory neurons and affects excitatory neurons indirectly through dis-inhibition. There are presumably different top-down signals in the brain that may originate from different brain areas and preferentially target different cell types. Therefore, we do not consider this an overly restrictive assumption. There is likely other top-down feedback to excitatory neurons, which we did not address here. An alternative implementation to interneuron-mediated feedback could be that dis-inhibitory circuits provide a gating signal for excitatory top-down feedback, which would otherwise be canceled out by inhibition. \n"}]

Author rebuttal:
We’d like to thank all reviewers for their helpful and constructive comments on our manuscript.  We have responded to each of you individually in our rebuttals below. Based on their collective feedback, we have performed additional simulations which we think strengthen the manuscript. We discuss their  outcomes in the following and we include new tables and figures in the attached PDF document.

### 1. Inhibitory control works with a simpler controller 

Several reviewers pointed out that our feedback weights were implausible because they changed at every timestep, i.e. $Q_i(t) = \frac{\partial r_L}{\partial u^I_i(t)}$. We agree that this may look like too much of a constraint. 
To address this, we derived the feedback weights from the network Jacobian at the uncontrolled (i.e. $c=0$) steady state. Specifically, we set $Q_i$ proportional to $\frac{\partial r_L}{\partial \tilde u^I_i}$ where $\tilde u^I_i$ are the inhibitory membrane potentials of layer $i$ at the uncontrolled steady-state. Thus, the feedback weights are static within a single trial. We reproduced  all our simulations on computer vision benchmarks in this setting and summarized the results in Table 1 of the attached PDF. All our results still hold in this setting. In fact, using these stationary feedback weights sped up our simulations due to reducing computational demand and slightly increased performance of our learning algorithm. We thus plan to replace Table 1 in the final manuscript with these new results with $n=10$ independent runs each. 

Using the open-loop steady state Jacobian as described removes the temporal dynamics of the feedback weights. However, the Jacobian is still input-dependent. Although our focus was not to solve the control problem, we acknowledge that related work used a similar feedback control mechanism (Meulemans 2021, 2022) and demonstrated learning with feedback weights that are not input-dependent and instead reflect the *average* transpose of the Jacobian across the dataset. To test whether our model would also support this static feedback weight regime, we performed the same simulations as in Table 1, but averaged the calculated feedback weights for each mini-batch. The results of these simulations are summarized in Table 2 of the attached PDF. Such feedback weights reflect the *average* Jacobian. They could be learned online from noise correlations, as shown by Meulemans et al. (2021). 

Together these findings indicate that our model is robust to the details of the feedback weights. We plan to include these results in the final manuscript. Furthermore, we expect that existing online learning algorithms for the feedback weights could be adapted to our dis-inhibitory control model in the future and will discuss this possibility in the revised discussion section.

### 2. More comprehensive comparison of in-vitro slice experiments with previous models

Some reviewers were concerned that we did not clarify sufficiently how our experimental predictions differ from existing bio-plausible credit assignment models. We agree that we could have done a better job at describing the commonalities and differences.

To address this point, we repeated the same experiments for the microcircuits and associated learning rules presented in related papers on credit assignment. Specifically, we compare to dendritic errors rules, (PDF Fig. 1a–c; Gilra & Gerstner (2018), Urbanczik & Senn (2014), Sacramento et al. (2018)),deep feedback control (PDF Fig. 1d,e; Meulemans et al. (2021 & 2022)) and burst multiplexing (PDF Fig. 1f,g; Payeur et al. (2021), Greedy et al. (2022)).Most models cannot reproduce the experimentally observed BCM-like learning curve. While burst-multiplexing models could reproduce a learning rule with BCM-like moving threshold if output spikes are Poisson distributed, the proposed underlying mechanism requires a spatially well-separated apical dendrite, specific to cortical L5 neurons. It is, therefore, difficult to see how L2/3 cortical neurons or, for instance, neurons in the basolateral amygdala could use a similar mechanism. In both cases, however, prominent disinhibitory circuit motifs are known to exist in neurobiology. In reality it is entirely conceivable that both burst-multiplexing and inhibitory control work side-by-side. We will clarify these commonalities and differences by adding a discussion and similar comparison along with a summarizing figure in the supplementary material of our final manuscript.
	
### 3. Preliminary results on learning with more biologically plausible inhibitory connectivity 

Some reviewers raised a concern about the plausibility of the one-to-one inhibitory connectivity. We agree with the reviewers that this connectivity is  biologically implausible. We made this simplifying model choice because it allowed us to directly relate the model to existing frameworks with neuron-specific error signals. Reducing the number of interneurons to reflect realistic ratios requires feedback to be low-dimensional which opens up new exciting research questions beyond the scope of the present article. We have started to empirically test the notion of learning with low-dimensional feedback and obtained promising preliminary results (see Figure 2 of the attached PDF). Specifically, we investigated training ANNs with low-rank feedback weights in a Direct Feedback Alignment setting  (Nøkland, 2016). While these results indicate that learning with low-dimensional feedback (and thus realistic number of interneurons) signals is possible, relating these findings to a recurrent architecture compromising excitatory and inhibitory neurons requires additional work that is beyond the scope of the current paper.



Meta-review: This paper proposes a model of learning in cortical circuits that relies on Hebbian plasticity paired with disinhibition driven by error signals. The authors show that the model accounts for plasticity data both in the presence and absence of inhibition. They also show that the model compares comparably to backpropagation on some simple non-linear tasks. The model makes some specific predictions about inhibitory modulation of excitatory plasticity.

The reviewers felt that the paper was interesting, though they raised some concerns with respect to the novelty of the contributions, the robustness of the comparisons to experimental data, and some remaining issues of biological implausibiltiy. After rebuttals, the reviewers were largely satisfied, and all had scores above the acceptance threshold, so a decision of accept was reached.
Decision: Accept (poster)