SlowGuess/NeurIPS_2023_OCR huggingface dataset:
where  $\mathrm{pre}_j$  is presynaptic neuronal activity,  $g(\mathrm{post})_i$  is a nonlinear function  $g(.)$  of postsynaptic neuronal activity and  $e_i$  is a neuron-specific error signal that determines the sign of plasticity. In this section, we showcase how error-modulated plasticity rules drive neuronal activity to a stable fixed point and discuss their predictions for in vitro electrophysiology experiments (Supplementary Fig. S1).
Dendritic error rules
Deep Feedback Control
Figure S1: Simulated in vitro patch clamp experiments with different learning rules proposed for bio-plausible credit assignment. (a) Dendritic error rules [1] encode the error (red) in the difference between somatic and basal dendritic voltage. When top-down feedback nudges neuronal activity towards a target, this results in error-modulated learning with a stable fixed point at  $r^{\mathrm{E}} = r^{\mathrm{tar}}$ . (b) In the absence of a closed-loop feedback circuit, dendritic error rules postulate that the dendritic potential is canceled out by recurrent inhibition, which would result in weight changes limited to LTD. (c) When isolating a single neuron without recurrent connections from excitatory to inhibitory neurons, no weight change can take place except by experimentally injecting currents into dendritic compartments. Increasing inhibitory activity experimentally should thus cause LTD. (d) DFC models [2, 3] use a dendritic error learning rule that compares dendritic (only feed-forward) to somatic (including feedback) activity. In the presence of a top-down controller, this leads to error-modulated learning. (e) Similar to (c), in the absence of top-down feedback, the sign of plasticity is determined by externally injected current into the somatic compartment. (f) In models using burst-dependent plasticity [4, 5], the error (red) is encoded in the deviation of a baseline apical membrane potential, which determines the burst probability of the neuronal output. In a closed-loop circuit, this leads to error-modulated learning. (g) If firing rates are Poisson distributed and thus burst probability increases as a function of firing rate, such models can resemble plasticity with a postsynaptic threshold. In this adapted model, increasing inhibitory currents into the apical compartment can bias the learning rate towards LTD.
Dendritic error rules. Dendritic error rules employ neuron models with multiple segregated compartments to encode a neuron-specific error signal  $e_{i}$  in the difference between neuronal activity of different compartments [1, 6]. Here, we implement the model described by Sacramento et al. [1], in which the error is encoded in the difference between a somatic voltage  $u_{\mathrm{S}}$  and a basal dendritic voltage  $u_{\mathrm{B}}$ . For simplicity, we consider a single pyramidal neuron receiving presynaptic input  $x$  weighted by a feed-forward weight  $w$ . Top-down feedback provides a current to an apical dendritic compartment  $u_{\mathrm{A}}$  that is canceled out by recurrent inhibition, leaving an error signal in the apical dendrite that in turn influences the somatic voltage (Supplementary Fig. S1a). The error signal used
for the bottom-up weight update is then decoded from the difference between the output of the neuron,  $r = \phi(u_{S})$ , and the output that would be observed if the apical dendrite potential (i.e. the top-down error) was zero. Specifically, the weight change at a synapse with presynaptic input  $x$  is given by
$$
\Delta w ^ {\text {D e n d r . E r r o r}} = \eta (\phi (u _ {\mathrm {S}}) - \phi (\hat {u} _ {\mathrm {B}})) x \tag {13}
$$
where  $\hat{u}_b$  is a function of the dendritic potential that takes into account dendritic attenuation parameters of the model,  $\phi(.)$  is a nonlinear activation function and  $\eta$  is a learning rate (see Sacramento et al. [1] for a detailed description of the model dynamics).
When this microcircuit is intact, top-down feedback is proportional to a target firing rate  $r^{\mathrm{tar}}$  and the neuron-specific error  $e_i$  determining the sign of plasticity is proportional to the output error  $r^{\mathrm{tar}} - r$ . Consequently, the synaptic weight will increase when the neuronal firing rate is below the target rate and vice versa (Supplementary Fig. S1a).
In contrast, when the apical dendrite does not receive top-down inputs, for example when top-down afferents are severed in an in vitro slice preparation, the plasticity rule in Eq. (13) cannot decode an error. In this setting, recurrent inhibition would still try to cancel out an expected top-down input to the apical compartment, leading to a negative apical potential and in turn to LTD (Supplementary Fig. S1b). Note that in other implementations of dendritic error rules, that do not employ a recurrent inhibitory circuit [6, 7],  $\Delta w = 0$  in this setting.
If the local inhibitory microcircuit is also impaired, for example by blocking recurrent connections from excitatory neurons to inhibitory neurons, the apical voltage is independent of presynaptic stimulation. In this isolated neuron setting, which arguably resembles most experimental conditions for in vitro electrophysiology, dendritic error rules would predict no weight change, except if current is injected into the apical compartment experimentally, in which case the weight change would be proportional to the injected current (Supplementary Fig. S1c)
Deep Feedback Control. The DFC framework put forward by Meulemans et al. [2, 3] also makes use of a dendritic error rule, but considers a simpler microcircuit without recurrent inhibition. The weight change in DFC is proportional to the difference between a neuron's bottom-up inputs, encoded in a basal dendritic potential  $u_{\mathrm{B}}$ , and its somatic potential  $u_{\mathrm{S}}$ . The somatic potential integrates both bottom-up inputs and a top-down control signal that moves the neuronal firing rate towards the target  $r^{\mathrm{tar}}$ . The DFC update rule is defined as
$$
\begin{array}{l} \Delta w ^ {\mathrm {D F C}} = \eta \left(u _ {\mathrm {S}} - u _ {\mathrm {B}}\right) x (14) \\ = \eta (u _ {\mathrm {S}} - w x) x. (15) \\ \end{array}
$$
When the top-down control loop is intact, the DFC learning rule resembles the learning dynamics of dendritic error rules and creates a stable fixed point at  $r = r^{\mathrm{tar}}$  (Supplementary Fig. S1d). As was the case in dendritic error rules, when the top-down control signal is impaired, the sign and magnitude of the weight change are proportional to externally injected currents (Supplementary Fig. S1e).
Burst-dependent plasticity. Payeur et al. [4] suggested an alternative mechanism to leverage segregated neuronal compartments for error-driven plasticity. As in dendritic error rules, burst-dependent plasticity relies on top-down feedback connections to an apical dendritic compartment. However, in contrast to the previously described learning rules, burst-dependent plasticity does not rely on compute  $e_i$  using differences in neuronal activity between compartments. Instead, burst-dependent plasticity exploits the unique spiking behavior of cortical layer 5 pyramidal neurons [8], in which inputs to the apical dendrite determine whether bottom-up inputs to the neuron result in a single output spike ("event") or several output spikes fired in quick succession ("burst"). This approach allows the neuron to multiplex two streams of information in its firing rate: bottom-up forward signals are encoded in the event rate  $r_{\text{event}}$  while top-down errors are encoded in the burst probability  $p$ . To illustrate the dynamics arising from burst-dependent plasticity, we consider the rate-based version of the model proposed by Greedy et al. [5], for which the weight change is defined as
$$
\Delta w ^ {\text {B u r s t - d e p e n d e n t}} = \eta r _ {\text {e v e n t}} (p - p _ {b}) x \tag {16}
$$
where  $p_b$  is a parameter denoting the baseline burst probability. The sign of plasticity is thus determined by deviations from the baseline burst probability (see Greedy et al. [5] for details on model dynamics).
In the intact, closed-loop microcircuit, top-down feedback communicates an error signal to the apical compartment that in- or decreases the burst probability depending on whether the neuronal output  $r_{\mathrm{event}}$  is smaller or larger than the target rate  $r^{\mathrm{tar}}$ . Similar to other error-modulated plasticity rules, this creates a stable fixed point for plasticity at  $r_{\mathrm{event}} = r^{\mathrm{tar}}$  (Supplementary Fig. S1f).
Because the baseline burst probability  $p_b$  acts like a postsynaptic plasticity threshold, burst-dependent plasticity can resemble some aspects of phenomenological Hebbian plasticity rules. Specifically, when synaptic inputs to the neuron follow a Poisson distribution over time, its output will also be Poisson distributed and the burst probability is not solely determined by top-down apical inputs. To illustrate this scenario, we consider an isolated neuron without top-down feedback to its apical dendrite. Given Poisson-distributed inputs  $x$ , we can assume that the burst probability of the postsynaptic neuron is determined by the input Poisson rate, i.e.  $p = f(x)$ . Plotting the weight change as a function of the input Poisson rate while keeping the baseline burst probability  $p_b$  fixed results in a learning rule that resembles phenomenological Hebbian plasticity with a postsynaptic threshold (Supplementary Fig. S1g).
Appendix B Analysis of the learning rule for dis-inhibitory control
Let's first recall the learning rule expressed in Eq. (9), where local error is encoded in the difference between the excitatory firing rate and inhibitory membrane potential. To enhance clarity, we reformulate the weight change for a singular synapse,  $w_{ij}$ , established between presynaptic excitatory neuron  $j$  and postsynaptic excitatory neuron  $i$ :
$$
\Delta w _ {i j} = \phi^ {\prime} \left(u _ {i} ^ {\mathrm {E}}\right) \left(r _ {i} ^ {\mathrm {E}} - u _ {i} ^ {\mathrm {I}}\right) r _ {j} ^ {\mathrm {E}}. \tag {17}
$$
where we assume all quantities are evaluated at equilibrium state. Here,  $r_j^E$  and  $r_i^E$  represent the pre- and post-synaptic firing rates respectively.  $u_i^E$  symbolizes the postsynaptic membrane potential, whereas  $u_i^I$  denotes the membrane potential of the inhibitory neuron linked to the postsynaptic cell. This learning rule is non-local since the postsynaptic neuron doesn't possess direct access to the membrane potential of an inhibitory neuron  $u_i^I$ .
To navigate around this limitation, we introduced a linear approximation of the inverse inhibitory activation function in Eq. (11), yielding a local learning rule of the generic form:
$$
\Delta w _ {i j} = \phi^ {\prime} \left(u _ {i} ^ {\mathrm {E}}\right) \left(r _ {i} ^ {\mathrm {E}} - \theta - \delta r _ {i} ^ {\mathrm {I}}\right) r _ {j} ^ {\mathrm {E}}. \tag {18}
$$
The simplified microcircuit model we studied here assumes a particular form of one-to-one connectivity in which each excitatory neuron only receives input from one inhibitory neuron and vice-versa. Conceptually,  $r_i^I$  can thus be interpreted as the inhibitory current received by the postsynaptic neuron.
B.1 Relationship between  $\mathcal{L}$  and  $\mathcal{H}$
Following [3], it is straight forward to show that a reduction in  $\mathcal{H}$  leads to a reduction in  $\mathcal{L}$  by observing that, if  $\mathcal{H}$  is minimized to zero,  $\mathcal{L}$  is also minimized to zero. This is simple to show by observing that  $\mathcal{H} = 0$  only occurs when  $\mathcal{L} = 0$  (cf. Eq. (4)), i.e. when the uncontrolled network output equals the target. Formally, for any given loss that evaluates to zero if  $\mathbf{r}_L = \tilde{\mathbf{r}}_L^{\mathrm{tar}}$ , i.e.  $\mathcal{L}(\mathbf{x},\mathbf{x}) = 0$ , we have
$$
\mathcal {H} = \sum_ {n} \| Q \tilde {\mathbf {c}} ^ {*} (n) \| _ {2} ^ {2} = 0 \Longleftrightarrow \sum_ {n} \mathcal {L} (\tilde {\mathbf {r}} _ {L} (n), \mathbf {r} _ {L} ^ {\text {t r u e}} (n)) = 0 \tag {19}
$$
where  $\tilde{\mathbf{r}}_L$  denotes the output of the network in the absence of a controller. However, in contrast to the networks considered in [3], the dis-inhibitory controller considered in this work is not guaranteed to minimize the surrogate loss  $\mathcal{H}$ , and thus  $\mathcal{L}$  to zero.
B.2 Linear approximation of the inverse function
The learning dynamics described by the learning rule in Eq. (11) depend implicitly on the chosen nonlinear activation function  $r = \phi(u)$ , which dictates the error induced by the linear approximation. For all simulations in this work, we use a soft rectifying nonlinearity as neuronal activation function:
$$
\phi (u) = \beta \log (1 + \exp (u - \gamma)) \tag {20}
$$
which is parameterized through its scale  $\beta$  and shift  $\gamma$ . Additionally, the learning dynamics are contingent on the choice of linearization parameters  $\theta$  and  $\delta$  used to approximate the inverse activation function,
$$
\phi^ {- 1} (r) = \gamma + \log \left(\exp \left(\frac {r}{\beta}\right) - 1\right) \approx \theta + \delta r. \tag {21}
$$
Fig. S2 depicts functionally and phenomenologically different learning dynamics in the same three settings we already investigated in Section 4, that can be obtained by changing the linearization parameters.
Figure S2: Effect of varying linearization parameters  $\theta$  and  $\delta$  on the learning rule in different microcircuit conditions. The first column depicts the inverse activation function Eq. (21) and several linear approximations thereof, parameterized through the intercept  $\theta$  and slope  $\delta$ . The second column depicts the dynamics of the resulting learning rule in a microcircuit under the influence of a top-down disinhibitory controller. The third column depicts the learning dynamics in the absence of top-down control. The last column depicts the learning dynamics under simulated experimental conditions, with direct control over the inhibitory current.
For the training of the deep neural networks discussed in Section 5, we adopted a practical approach and linearize the inverse function using a first-order Taylor expansion. This method effectively reduces the linear approximation to a single hyperparameter, the linearization point  $\tilde{r}$ . Therefore, we approximate the inverse activation function as follows:
$$
\phi^ {- 1} (r) = f (r) \approx f (\tilde {r}) + f ^ {\prime} (\tilde {r}) (r - \tilde {r}), \tag {22}
$$
where  $f^{\prime}(\tilde{r})$  denotes the derivative of the inverse activation function evaluated at the linearization point  $\tilde{r}$ . From this linearization, we can extract the parameters  $\theta$  and  $\delta$  in relation to  $\tilde{r}$ :
$$
\theta = f (\tilde {r}) - \tilde {r} f ^ {\prime} (\tilde {r}) \quad \delta = f ^ {\prime} (\tilde {r}). \tag {23}
$$
Denoting  $\tilde{u} = f(\tilde{r}) = \phi^{-1}(\tilde{r})$ , we can use the inverse function theorem to formulate the relationship between the linearization point  $\tilde{r}$  and the general linearization parameters,  $\theta$  and  $\delta$ , in a more interpretable fashion:
$$
\theta = \tilde {u} - \frac {\tilde {r}}{\phi^ {\prime} (\tilde {u})} \quad \delta = \frac {1}{\phi^ {\prime} (\tilde {u})}. \tag {24}
$$
Appendix C Simulation details
All numerical simulations were implemented using Python 3.8.10 and were executed on NVIDIA Quadro RTX 5000 GPUs. The software stack includes Jax [9], Flax [10], and Diffrax [11]. Datasets were obtained from the TensorFlow datasets library [12]. The code to reproduce our results is publicly available at https://github.com/fmi-basel/disinhibitory-control.
C.1 Single-synapse experiments
For the investigation of learning dynamics at a single synapse (Section 4), we simulated a single microcircuit unit. The microcircuit unit was driven by increasing the strength of afferent input into the excitatory neuron. To capture the input-output curve of neurons in the absence of any background inputs, we parameterized the neuronal activation function Eq. (20) with  $\beta = 1.0$  and  $\gamma = 3.0$  for both excitatory and inhibitory neurons. For each input strength, the microcircuit was simulated for  $600~\mathrm{ms}$  to ensure that neuronal dynamics settled to an equilibrium. Synaptic weight updates where computed at the equilibrium state. Numerical integration was performed using the forward Euler method with a time step of  $1\mathrm{ms}$ .
C.2 Student-teacher task
Inputs. For the student-teacher learning task (Section 5), we generated 30 sine wave inputs with random frequency, amplitude and phase shift. Frequencies were randomly drawn from a uniform distribution  $\mathcal{U}(0.1\mathrm{Hz},2.0\mathrm{Hz})$ . Similarly, sine amplitudes were randomly drawn from the distribution  $\mathcal{U}(0.1,1.0)$  and phase shifts were drawn from the distribution  $\mathcal{U}(0,\pi)$ . Sine waves were continuously generated and used as input to both the student and teacher network during training. Evaluation for panel B in Fig. 4 before and after training was performed on a 5-second window of the training data.
Feedback weights. Feedback weights  $\mathbf{Q}_i$  were initialized proportional to the normalized network Jacobian at the beginning of training:
$$
- \mathbf {Q} _ {i} (0) = \alpha \frac {\mathbf {J} _ {i} (0)}{\| \mathbf {J} _ {i} (0) \| _ {F}} \tag {25}
$$
where we normalized the Jacobian by dividing it by its Frobenius norm. The parameters  $\alpha$  controls the norm of the effective Jacobian, and thus the magnitude of the feedback weights. We found that normalizing the Jacobian is not necessary for good training performance, but prevents exorbitantly large values in the feedback weights and can sometimes speed up numerical integration of the neuronal dynamics and aid the stability of the dynamical system.
As the forward weights  $\mathbf{W}_i$  change over the course of learning, the network Jacobian of the student network relevant for computing suitable feedback weights changes as well. To account for the learning-induced changes in the network Jacobian, we make the feedback weights  $\mathbf{Q}_i$  part of the
network dynamics and continuously nudge them towards the normalized network Jacobian with a slow time constant  $\tau_{Q}$ :
$$
\tau_ {Q} \frac {d \mathbf {Q} _ {i}}{d t} = - \mathbf {Q} _ {i} (t) + \alpha \frac {- \mathbf {J} _ {i} (t)}{\| - \mathbf {J} _ {i} (t) \| _ {F}} \tag {26}
$$
As a result, feedback weights are dynamically adjusted to retain good control performance. We chose  $\tau_{Q}$  to be faster than the plasticity time constant  $\tau_{W}$  and much slower than changes in the input  $x(t)$ . As a result, the feedback weights reflect the average network Jacobian over time, but adapt to changes in the neuronal dynamics caused by feed-forward plasticity. Meulemans et al. [2, 3] have demonstrated that feedback weights reflecting the average Jacobian can also be learned using a simple, fully local anti-Hebbian learning rule based on noise correlations between neuronal activity and top-down control.
Training. Both student and teacher networks were randomly initialized using Kaiming initialization and hyperparameters relating to numerical integration and neuronal dynamics were kept identical for student and teacher networks (see Table S1). Student networks were trained continuously for a total of 5 minutes, split into 60 periods each lasting 5 seconds. For each 5-second window, we first computed the output of the teacher network to obtain a target for the top-down controller. We then computed the output of the student network in an open-loop setting without top-down control to calculate the MSE loss  $\mathcal{L}$ . Finally, we simulated the student network with active top-down control and continuously monitored the least-control loss  $\mathcal{H}$ . Numerical integration was performed using Tsitouras' 5th order explicit Runge Kutta method [13] with an adaptive step size.
Table S1: Hyperparameters used for training networks on the student-teacher task (Fig. 4).
Parameter	Description	Value
τE	Exc. membrane time constant	20 ms
βE	Exc. activation function scale	5.0
γE	Exc. activation function shift	3.0
τI	Inh. membrane time constant	5 ms
βE	Inh. activation function scale	1.0
γE	Inh. activation function shift	3.0
τc	Controller time constant	100 ms
kp	Proportional control strength	30.0
ki	Integral control strength	15.0
τW	Weight update time constant	60 s
τQ	Feedback weight update time constant	30 s
r	Linearization point for learning rule	0.5
α	Jacobian norm. constant	7.5
C.3 Computer vision benchmarks
Data preprocessing and network architecture. MNIST and Fashion-MNIST images were rescaled to values between 0 and 1 and flattened before being used as inputs to the networks. Each hidden layer was composed of 256 excitatory-inhibitory microcircuit units for networks trained with disinhibitory control or 256 neurons for networks trained with BP.
Classification with Softmax and cross-entropy loss. Conventionally, classification tasks are solved using a Softmax layer as output of the network. The Softmax layer is trained to match a one-hot encoded label  $\mathbf{r}_L^{\mathrm{tar}}$ . As pointed out by Meulemans et al. [3], because the Softmax layer output only matches the one-hot vector when it has an infinite input, this can lead to problems when feedback control is strong. Thus, we use soft targets where the value of  $\mathbf{r}_L^{\mathrm{tar}}$  is 0.99 for the correct target and  $0.01 / n_L - 1$  for all other entries. Following [3], we use a linear output layer and absorb the Softmax operation into the cross-entropy loss function:
$$
\mathcal {L} ^ {\text {c l a s s i f i c a t i o n}} = - \sum_ {n} \mathbf {r} _ {L} ^ {\operatorname {t a r}} (n) \log (\operatorname {S o f t m a x} (\mathbf {r} _ {L} (n))) \tag {27}
$$
Our dis-inhibitory controller and the associated learning rules require a nonlinear activation function and recurrent inhibition, which could interfere with the Softmax in the loss function. Thus, we
Table S2: Hyperparameters used for training networks on computer vision benchmarks
Parameter	Description	Value
τE	Exc. membrane time constant	20 ms
τI	Inh. membrane time constant	5 ms
β	Activation function scale	1.0
γ	Activation function shift	0.0
τc	Controller time constant	100 ms
kp	Proportional control strength	0.2
ki	Integral control strength	0.4
α	Jacobian norm. constant	1.0
implemented the readout layer for classification tasks as a linear layer without excitatory-inhibitory units:
$$
\tau_ {E} \frac {d}{d t} \mathbf {r} _ {L} = - \mathbf {r} _ {L} (t) + \mathbf {W} _ {L} \mathbf {r} _ {L - 1} ^ {E} (t) + \mathbf {Q} _ {L} \mathbf {c} (t) \tag {28}
$$
with  $\mathbf{Q}_L = I$ , and performed weight updates according to the deep feedback control learning rule [3].
Hyperparameter optimization. Optimal values for the controller parameters  $k_{\mathrm{p}}$  and  $k_{\mathrm{i}}$  were obtained using the Optuna package [14]. Optimization was performed using a Tree-structured Parzen Estimator [15] with 25 starts. Specifically, hyperparameters were chosen to minimize the validation loss of a 3-layer network after 20 training epochs using the exact inverse learning rule on the MNIST dataset.
Training. Network weights were initialized using Xavier initialization and trained for 50 epochs with a batchsize of 100. For every input sample, we first let the network settle to an equilibrium in the absence of feedback control. This uncontrolled equilibrium state was used to calculate the feedback weights  $\mathbf{Q}$  and used as initial state for the second convergence with top-down control. Numerical integration was performed using Tsitouras' 5th order explicit Runge-Kutta method [13] with an adaptive step size for a maximum duration of 2 seconds or until equilibrium was reached, defined by an absolute tolerance value of  $10^{-6}$ . Synaptic weight updates were performed at the controlled equilibrium state and averaged across each minibatch. For all experiments, we used the ADAM optimizer with a learning rate 0.001.
Feedback weights. Feedback weights were calculated proportional to the normalized network Jacobian: