Network and learning setup
The detailed governing equations of our setup can be found in Methods (Appendix A ). We consider a RNN with N_{in} input units, N hidden units and N_{out} readout units (Figure 1 A). We verified that trends hold for different network sizes and refer the reader to Appendix A.3 for more details. The update formula for h_{t}\in\mathbb{R}^{N} (the hidden state at time t ) is governed by:

h_{t+1}=\phi(W_{h}f(h_{t}),W_{x}x_{t}), (1)

where \phi(\cdot):\mathbb{R}^{N}\rightarrow\mathbb{R}^{N} is the hidden state update function, f(\cdot):\mathbb{R}^{N}\rightarrow\mathbb{R}^{N} is the activation function, W_{h}\in\mathbb{R}^{N\times N} (resp. W_{x}\in\mathbb{R}^{N_{in}\times N} ) is the recurrent (resp. input) weight matrix and x\in\mathbb{R}^{N_{in}} is the input. For \phi , we consider a discrete-time implementation of a rate-based recurrent neural network (RNN) similar to the form in [ 137 ] (details in Appendix A ). Readout \hat{y}\in\mathbb{R}^{N_{out}} , with readout weights w\in\mathbb{R}^{N_{out}\times N} , is defined as

\hat{y}=\langle w,f(h_{t})\rangle

We performed experiments on three tasks: sequential MNIST [ 138 ] , pattern generation [ 139 ] and delayed match-to-sample tasks [ 140 ] .
The objective is to minimize scalar loss L\in\mathbb{R} , which is defined as

L(W_{h})=\begin{cases}\frac{1}{2TB}\sum_{i=1}^{B}\sum_{t=1}^{T}\sum_{k=1}^{N_{out}}(\hat{y}^{(i)}_{k,t}-y^{(i)}_{k,t})^{2}, for regression tasks\cr\frac{-1}{TB}\sum_{i=1}^{B}\sum_{t=1}^{T}\sum_{k=1}^{N_{out}}\pi^{(i)}_{k,t}log\hat{\pi}^{(i)}_{k,t}, for classification tasks\end{cases} (3)

given target readout y\in\mathbb{R}^{N_{out}} , task duration T\in\mathbb{R} and batch size B\in\mathbb{R} . \pi_{k,t}\in\mathbb{R} is the one-hot encoded target for readout unit k at time t and \hat{\pi}_{k,t}=\text{softmax}_{k}(\hat{y}_{1,t},\dots,\hat{y}_{N_{OUT},t})=\exp(\hat{y}_{k,t})/\sum_{k^{\prime}}\exp(\hat{y}_{k^{\prime},t}) is the predicted category probability.

Different learning algorithms examined in this work are BPTT (our benchmark), which update weights by computing the exact gradient ( \nabla L(W_{h})\in\mathbb{R}^{N\times N} ):

\Delta W_{h}=-\eta abla L(W_{h}), (4)

and three SoTA bio-plausible learning rules that update weights using approximate gradient:

\widehat{\Delta W_{h}}=-\eta\tilde{ abla}L(W_{h}), (5)

where \tilde{\nabla}L(W_{h})\in\mathbb{R}^{N\times N} denotes a gradient approximation and \eta\in\mathbb{R} denotes the learning rate. These three learning rules are explained further in Appendix A (Methods) but we note that these bio-plausible learning rules are based on truncations of dependency paths — on the computational graph for the exact gradient— that are biologically implausible to compute (Figure 1 A). In all figures, learning rules are labeled as "Three-factor" (symmetric e-prop), "RF Three-factor" (RFLO) and "MDGL", respectively. We remark that the focus here is on comparing artificial to bio-plausible learning rules, rather than between biological rules. Finally, we note that tasks were learned with mostly comparable training accuracies for all learning rules, and that generalization gaps reflect a testing departure from these values. We refer the reader to the Appendix for more details (Appendix A.3 ).