SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
- \mathbf {Q} _ {i} = \alpha \frac {\tilde {\mathbf {J}} _ {i}}{\| \tilde {\mathbf {J}} _ {i} \| _ {F}} \tag {29}
$$
where  $\tilde{\mathbf{J}}_i$  is the Jacobian obtained at the uncontrolled equilibrium state (cf. Eq. (7)). Because this calculation results in input-dependent feedback weights, which are biologically implausible, we also consider the case in which feedback weights reflect the average Jacobian. As demonstrated in Meulemans et al. [2, 3], such feedback connections could be learned locally making use of noise correlations between neuronal compartments receiving top-down feedback and the strength of the control signal. Here, we instead take a practical approach and take the average of the Jacobian across each minibatch:
$$
- \mathbf {Q} _ {i} ^ {(\text {A v g J})} = \alpha \frac {\tilde {\mathbf {J}} _ {i} ^ {\text {A v g}}}{\| \tilde {\mathbf {J}} _ {i} ^ {\text {A v g}} \| _ {F}} \tag {30}
$$
$$
\tilde {\mathbf {J}} _ {i} ^ {\text {A v g}} = \frac {1}{N} \sum_ {n = 1} ^ {N} \tilde {\mathbf {J}} _ {i} \tag {31}
$$
Table S3: Validation accuracy in % for networks trained with BP or dis-inhibitory feedback control. Reported values are mean +/- stdev  $(n = 10)$
MNIST	Fashion-MNIST
1 hidden layer	3 hidden layers	1 hidden layer	3 hidden layers
BP	98.0 ± 0.24	98.2 ± 0.14	90.1 ± 0.29	90.3 ± 0.34
Exact inverse	97.5 ± 0.19	97.5 ± 0.19	90.0 ± 0.24	90.0 ± 0.38
Linear threshold	96.9 ± 0.16	96.4 ± 0.18	88.8 ± 0.25	88.1 ± 0.25
Exact inverse (Avg J)	97.6 ± 0.15	96.9 ± 1.6	89.8 ± 1.39	89.2 ± 0.48
Linear threshold (Avg J)	96.4 ± 0.15	95.8 ± 0.21	87.9 ± 0.36	86.2 ± 0.52