H OW GRADIENT ESTIMATOR VARIANCE AND BIAS COULD IMPACT LEARNING IN NEURAL CIRCUITS
A BSTRACT
There is growing interest in understanding how real brains may approximate gradients and how gradients can be used to train neuromorphic chips.
However, neither real brains nor neuromorphic chips can perfectly follow the loss gradient, so parameter updates would necessarily use gradient estimators that have some variance and/or bias.
Therefore, there is a need to understand better how variance and bias in gradient estimators impact learning dependent on network and task properties.
Here, we show that variance and bias can impair learning on the training data, but some degree of variance and bias in a gradient estimator can be beneﬁcial for generalization.
We ﬁnd that the ideal amount of variance and bias in a gradient estimator are dependent on several properties of the network and task: the size and activity sparsity of the network, the norm of the gradient and the curvature of the loss landscape.
As such, whether considering biologically-plausible learning algorithms or algorithms for training neuromorphic chips, researchers can analyze these properties to determine whether their approximation to gradient descent will be effective for learning given their network and task properties
I NTRODUCTION
Artiﬁcial neural networks (ANNs) typically use gradient descent and its variants to update their parameters in order to optimize a loss function (LeCun et al., 2015; Rumelhart et al., 1986).
Importantly, gradient descent works well, in part, because when making small updates to the parameters, the loss function’s gradient is along the direction of greatest reduction.
1 Motivated by these facts, a longstanding question in computational neuroscience is, does the brain approximate gradient descent (Lillicrap et al., 2020; Whittington & Bogacz, 2019)?
Over the last few years, many papers show that, in principle, the brain could approximate gradients of some loss function (Murray, 2019; Liu et al., 2021; Payeur et al., 2021; Lillicrap et al., 2016; Scellier & Bengio, 2017).
Also inspired by the brain, neuromorphic computing has engineered unique materials and circuits that emulate biological networks in order to improve efﬁciency of computation (Roy et al., 2019; Li et al., 2018b).
But, unlike ANNs, both real neural circuits and neuromorphic chips must rely on approximations to the true gradient.
This is due to noise in biological synapses and memristors, non-differentiable operations such as spiking, and the requirement for weight updates that do not use non-local information (which can lead to bias) (Cramer Benjamin et al., 2022; M. Payvand et al., 2020b; Laborieux et al., 2021;
Shimizu et al., 2021; Neftci et al., 2017; N. R. Shanbhag et al., 2019).
Thus, both areas of research could beneﬁt from a principled analysis of how learning is impacted by loss gradient variance and bias.
In this work, we ask how different amounts of noise and/or bias in estimates of the loss gradient affect learning performance.
As shown in a simple example in Fig.
9, learning performance can be insensitive to some degree of variance and bias in the gradient estimate, and even beneﬁt from it, but excessive amounts of variance and/or bias clearly hinder learning performance.
Results from optimization theory shed light on why imperfectly following the gradient—e.g. via stochastic gradient descent (SGD) or other noisy GD settings—can improve generalization in ANNs (Foret et al., 2020;
Chaudhari et al., 2019; Yao et al., 2018; Ghorbani et al., 2019).
However, most of these results treat unbiased gradient estimators. In contrast, in this work, we are concerned with the speciﬁc case of weight updates with intrinsic but known variance and bias, as is often the case in computational neuroscience and neuromorphic engineering.
Moreover, we also examine how variance and bias can hinder training, because the amount of variance and bias in biologically-plausible and neuromorphic learning algorithms is often at levels that impair, rather than improve, learning (Laborieux et al., 2021), and sits in a different regime than that typically considered in optimization theory.
The observations in Fig.
9 give rise to an important question for computational neuroscientists and neuromorphic chip designers alike: what amount of variance and bias in a loss gradient estimate is tolerable, or even desirable?
To answer this question, we ﬁrst observe how variance and bias in gradient approximations impact the loss function in a single parameter update step on the training data.
(We also extend this to multiple updates in Appendix A.)
We utilize an analytical and empirical framework that is agnostic to the actual learning rule, and derive the factors that affect performance in an imperfect gradient setting.
Speciﬁcally, we assume that each update is comprised of the contribution of the true gradient of the loss function with respect to the parameter, a ﬁxed amount of bias, and some noise.
Similar to Raman et al.
(2019), we derive an expression for the change in the loss function after a discrete update step in parameter space: w ( t +  t ) =
w ( t ) +  w ( t )  t , where  t is akin to learning rate in standard gradient descent algorithms.
We then characterize the impact on learning using the decrease in loss function under an approximate gradient setting as compared to the decrease in loss function when following the true gradient.
Our analysis demonstrates that the impact of variance and bias are independent of each other.
Furthermore, we empirically validate our inferences in ANNs, both toy networks, and various VGG conﬁgurations (Vedaldi & Zisserman, 2016) trained on CIFAR-10 (Krizhevsky & Hinton, 2009).
Our ﬁndings can be summarized as follows:
The impact of variance is second order in nature. It is lower for networks with a threshold non-linearity, as compared to parameter matched linear networks.
Under speciﬁc conditions, variance matters lesser for wider and deeper networks.
The impact of bias increases linearly with the norm of the gradient of the loss function, and depends on the direction of the gradient as well as the eigenvectors of the loss Hessian.
Both variance and bias can help to prevent the system from converging to sharp minima, which may improve generalization performance.
Altogether, our results provide guidelines for what network and task properties computational neuroscientists and neuromorphic engineers need to consider when designing and using noisy and/or biased gradient estimators for learning.
E STIMATING THE IMPACT OF VARIANCE AND BIAS ON TRAINING
Our analysis focuses on situations where the synaptic weights of a network, w , are trained to optimize a loss, L [ w ] .
We use an auxiliary variable, t , to denote different points in this optimization trajectory and  t to denote a single step in this trajectory.
(We extend to the multi-step case in Corollary A.2 of the Appendix.)
Compared to standard optimization protocols in the ANN literature,  t is akin to the learning rate.
Therefore, the loss as measured at one particular point in the trajectory is denoted as
L [ w ( t )] , and the loss at the next point in the trajectory, i.e., after a weight update step, is denoted by
L [ w ( t +  t )] .
In the rest of this work, we characterize L [ w ( t +  t )]  L [ w ( t )] as a function of the variance and bias in the weight update.
We assume that L is twice differentiable and  t is small enough such that L [ w ( t +  t )] can be effectively approximated around L [ w ( t )] using a second order Taylor series, i.e., higher order terms beyond the second order can be ignored.
Furthermore, we deﬁne a weight update equation that consists of the gradient, a ﬁxed bias, and some noise.
We denote the bias vector as b !  , where !  is a norm p N vector that indicates the direction of bias in the N -dimensional parameter space.
A special case of !  would be !  = !
1 when all parameters have the same bias.
Similarly, we assume white noise in the gradient estimates.
With these assumptions, we deﬁne the weight update equation to be:
[Equation]
where r w L [ w ( t )] denotes the ﬁrst derivative of the gradient of L evaluated at w ( t ) and ˆ n is a unit vector in the N -dimensional parameter space whose elements are zero-mean i.i.d.
(see Appendix
A for generality of this assumption), such that the total variance of  w ( t ) is  2 f ( N ) , where f ( N ) denotes the dependence of variance on the total number of network parameters, N .
Finally, we deﬁne  L t ( b,  2 ) as the change in L [ w ( t )] when performing the aforementioned weight update as compared to the change in L [ w ( t )] when the weight update step follows the true gradient, i.e.,
[Equation]
where [ L [ w ( t +  t )]  L [ w ( t )]] ( b,  2 ) is the change in loss with bias b and variance  2 .
For brevity, we drop the t is the subscript in the rest of the paper.
Formally, we present the following Lemma:
Lemma 2.1.
Second order Taylor series expansion.
Assuming a small learning rate, the bias, b , and variance,  2 , in gradient estimates can be linked to changes in L upon one weight update step as compared to the change in L under a true gradient descent weight update step:
[Equation]
where h · , · i denotes the dot product, Tr denotes the Trace operator of a square matrix, orange terms indicate the impact of bias, and the green term indicates the impact of variance.
In the next section, we present an in-depth analysis of the impact of variance when f ( N ) is sublinear, as is the case for some learning algorithms, e.g. Equilibrium Propagation (Laborieux et al., 2021).
We present a more general analysis along with the proofs of all lemmas and theorems in Appendix A.
2. A NALYSIS OF THE IMPACT OF VARIANCE AND BIAS ON TRAINING LOSS
One of the corollaries that follow from Lemma 2.1 is that the impact of variance and bias on the expected  L t ( b,  2 ) are independent of each other.
Moreover, it is clear that the impact of variance is directly proportional to the trace of r 2 w L [ w ( t )] , i.e., the loss Hessian, and inversely proportional to N , i.e., the number of trainable parameters in the network.
Previous work investigating the object categorization loss landscape for feedforward ANNs demonstrated a pronounced effect of width, i.e., increasing width leads to smoother loss landscapes (Li et al., 2018a), thereby leading to lower trace of Hessian (Hochreiter & Schmidhuber, 1994).
This empirical observation in deep ANNs implies that increasing the width both lowers the trace of the Hessian in addition to its obviously increasing the number of trainable network parameters.
Therefore, we can say directly from Lemma 2.1 that the impact of variance is lower for wider networks.
However, the role of depth is more complicated.
Notably, increasing depth (with no skip connections) could theoretically make the loss landscape less smooth (Li et al., 2018a).
Thus, we begin by analyzing the case of increasing depth in linear feedforward networks.
We prove that, in fact, increasing depth leads to lower values for Tr[ r 2 w L ] N , thereby implying a lower impact of variance.
Theorem 2.2.
Increasing depth lowers impact of variance For linear feedforward ANNs, the impact of variance on  L t (0 ,  2 ) for a ( L + 1) layer network is less than that of a L layer network, i.e.,
[Equation]
(4) where each layer has d units and weights initialized by the Xavier initialization strategy and the total variance in gradient estimates does not grow with the size of the network, i.e., f ( N ) is some constant
Although these assumptions on layer weights may not strictly hold over the course of training, empirical experience suggests that there exists a loose correspondence among these quantities if the conditions hold true during initialization.
Taken together, overparameterization in either width or depth leads to a lower impact of variance in gradient estimates on the network’s training.
Besides the network architecture, the non-linearity used in the network also impacts the function implemented by the network, and therefore affects the trace of the loss Hessian.
Speciﬁcally, we demonstrate that the trace of the loss Hessian is lower for networks with a threshold non-linearity with gain less than or equal to 1, such as a ReLU operation, as compared to a linear network.
Formally,
Theorem 2.3.
ReLU lowers impact of variance Let  ( . ) denote a threshold non-linearity function.
The impact of variance on  L t (0 ,  2 ) for a network with such non-linearities is less than that of an equivalent linear network, i.e.,
[Equation]
where the gain of  ( . ) is less than or equal to 1 (e.g. ReLU).
Summarily, the standard practices in deep neural networks, like overparameterized networks or the use of ReLU as a non-linearity, lead to a lower impact of variance on the network’s training.
Interestingly, brains are also arguably overparameterized and characterized by threshold non-linearities that lead to sparse activations.
These properties could offer a potential solution to countering any negative impact of variance in gradient estimates in the brain and allow biological agents to reliably train complex tasks despite not being able to actually conduct true gradient descent (Lillicrap et al., 2020).
It also suggests neuromorphic chips could tolerate more noise in gradient estimators if they are made larger.
In contrast to the impact of variance, the impact of bias is more nuanced and depends on the norm and direction of gradient, the direction of the bias vector, as well as the eigenvectors of the loss Hessian.
Using a few algebraic manipulations, Lemma 2.1 can be extended to show that:
Theorem 2.4.
The impact of bias on  L t ( b, 0) grows linearly with the norm of the gradient.
[Equation]
where P, Q are terms that are independent of the norm of the gradient, kr w L [ w ( t )] k .
This result implies that initialization and architectural considerations could play important roles in mitigating the impact of bias on training by limiting the norm of the gradient.
In the next section, we will empirically validate these results in linear and shallow networks trained on toy datasets as well as in VGG networks trained on CIFAR-10.
E MPIRICAL VERIFICATION OF THE VARIANCE AND BIAS RESULTS
Our experimental setup (illustrated in Fig. 2) is motivated by Raman et al.
and follows the standard student-teacher framework.
A “teacher” ANN is initialized with ﬁxed parameters.
The task is for a randomly initialized “student” network to learn the input-output mapping deﬁned by the teacher.
We defer the reader to the Appendix B for experimental details.
First, we validate our analytical results pertaining to the impact of variance, i.e., Theorems 2.2 and 2.
in relatively shallow (1-8 hidden layers) fully connected ANNs.
We ﬁx the teacher network architecture and use a ReLU non-linearity to threshold the intermediate layer activations.
Subsequently, we vary the depth and width of the student network with no non-linearity and observe  L t (0 ,  2 ) at different points in the loss landscape.
For each such setting, we add a ReLU non-linearity to the student network to validate the effect of a threshold non-linearity on  L t (0 ,  2 ) .
We plot the mean  L t (0 ,  2 ) across different points in the loss landscape in Fig.
Note that in order to use a sample accurately reﬂecting the loss landscape that would be encountered during a learning trajectory, we train a control network with the same architecture and non-linearity as the student network using the true gradient and evaluate  L t (0 ,  2 ) for each update step along the learning trajectory.
In doing so, we are able to observe the desired quantities across a wide range of gradient norm values
Furthermore, we validate Theorems 2.2 and 2.3 on VGG networks trained on the CIFAR-10 dataset.
Speciﬁcally, we maintain the student-teacher framework and ﬁx the teacher network to be a VGGnetwork trained on CIFAR-10 to 92.6% test accuracy, and we use different networks from the VGG family as student networks (with variance set to  2 = 20 ).
For each network, we use both random weight initialization and Imagenet-pretrained weights to determine whether initialization has an effect on the impact of variance.
Fig.
5 demonstrates that our results pertaining to depth and width hold for deep convolutional neural networks (CNNs).
Similarly, we validate Theorem 2.4, ﬁrst in shallow fully connected ANNs and subsequently in deep
CNNs trained on CIFAR-10.
Speciﬁcally, we demonstrate in Fig.
4 that the impact of bias (with b = 0 .
on performance of a linear shallow network grows with the norm of the gradient.
Note that the sign duality of  L t ( b, 0) in this ﬁgure follows from Theorem 2.4 where the quantities P and Q depend on the direction of the bias vector with respect to the direction of gradient and the eigenvectors of the loss Hessian (see proof in Appendix A for more details).
Therefore, bias can help or hinder learning depending on its direction.
This behaviour is in contrast to variance, which always hinders learning for a convex loss (Lemma 2.1).
Nonetheless, it is worth noting that when the gradient norm is small as bias increases it always hinders training (see Fig.
4B).
amount of bias when gradient norm is small (see ﬁrst term in equation).
Following this, we measure the absolute value of  L t ( b, 0) and plot its mean across multiple update steps in VGG networks trained on CIFAR-10.
In Fig.
6, we show that the relation stated in Theorem 2.4 holds across different VGG architectures, irrespective of the weight initialization.
Furthermore, deeper VGG networks have a lower impact of bias owing to a lower gradient norm.
Factor of downsampling indicates the reduction in the number of ﬁlters in convolutional layers and dimensionality of intermediate fully connected layers compared to the original VGG-19 conﬁguration.
E STIMATING THE IMPACT OF VARIANCE AND BIAS ON GENERALIZATION
In the previous section, we studied how variance and bias in gradient estimates impacts training.
However, from a machine learning perspective, an important question is to understand the impact on the system’s generalization.
Speciﬁcally, we ask the question: under what conditions could variance and bias in gradient estimates aid generalization performance?
Interestingly, current deep learning practices rely on not following the exact gradient of the loss function to train models, which have been demonstrated to help generalization performance.
Some common examples of such practices include stochastic gradient descent (SGD) and dropout 3 .
In this section, we use our framework to understand how variance and bias in gradient estimates could alter the learning trajectory, thereby impacting generalization.
Speciﬁcally, we investigate the conditions under which a parameter update would necessarily lead to descent on the loss landscape.
This matters for understanding generalization because the ﬂatness of the loss landscape is a good proxy for generalization (Baldassi et al., 2020; Jiang et al., 2019; Sankar et al., 2021; Tsuzuku et al., 2020; Petzka et al., 2021): ﬂatter minima tend to have better generalization performance than sharper ones.
We leverage this viewpoint to understand the impact of variance and bias on generalization.
Speciﬁcally, we investigate the conditions under which gradient approximations could help to avoid sharp minima, thereby promoting convergence to wide ﬂat minima.
3. A NALYSIS OF THE IMPACT OF VARIANCE AND BIAS DESCENT OF NARROW MINIMA
To quantify the ﬂatness of loss landscape, we use eigenvalues of the loss Hessian.
Without loss of generality, we assume that the loss Hessian is a normal square matrix, i.e., its eigenvectors form an orthogonal basis of the N -dimensional parameter space.
In order to understand the properties of loss minima that the network could potentially converge to, we derive the sufﬁciency conditions under which a parameter update ascends (or descends) the loss landscape.
This strategy indicates that noise in gradient estimates would lead to the system not descending the loss landscape when the minima is too sharp, which can aid generalization.
Theorem 3.1.
Noise helps avoid sharp minima
The sufﬁcient condition under which noise prevents the system from descending the loss landscape, i.e., L [ w ( t +  t )]  L [ w ( t )] , is
[Equation]
where  1   2 ...   N denote the eigenvalues of r 2 w L around the minima.
Interestingly, the condition presented in Theorem 3.1 presents a lower bound for the smallest eigenvalue, i.e., the eigenvalue associated with the direction of least curvature, and connects it to the variance to gradient norm ratio.
This ratio can be thought of as the inverse noise ratio when there is no bias in the approximation.
Furthermore, we also show in Appendix A that the sufﬁcient condition for descending the loss landscape is similar to the one presented in Theorem 3.1, but provides an upper bound for the largest eigenvalue, i.e.,  1  2  t [Equation]
Note that as we get closer to the minimum, the gradient norm would decrease (property of smooth L ), and therefore, the inverse noise ratio would increase.
Thus, the upper bound on curvature of loss minima that the system can potentially converge to decreases.
Taken together, this shows that variance helps the system avoid converging to sharp minima and instead promotes converging to wide ﬂat minima.
Bias, on the other hand, has a more nuanced effect on the learning trajectory.
This complexity is unsurprising given the dependence of the impact of bias on direction of the bias vector, the direction of the gradient, and the eigenvectors of the loss Hessian.
Once again, we derive the sufﬁciency condition for when a parameter update step would produce a decrease in the loss function.
It turns out this condition depends on both minima ﬂatness and amount of bias relative to the gradient norm.
Theorem 3.2.
Bias prevents descent into local minima dependent on Hessian spectrum.
Bias in gradient estimates could prevent converging to a minima.
Speciﬁcally, the sufﬁcient condition for the weight updates to produce a decrease in L when using a biased estimate of the gradient is:
[Equation]
where 2 = P i  2 i , i.e., the Frobenius norm of the loss Hessian, and  1   2 ...   N denote the eigenvalues of r 2 w L around the minima and r 2 w L is a normal matrix.
The condition in Theorem 3.2 indicates that a system that follows a biased gradient approximation will descend the loss landscape until some neighbourhood around a minimum (characterized by the minima ﬂatness and relative bias).
Inside this neighbourhood, the system may not descend the loss landscape and therefore not converge to the minimum.
The condition depends on minima ﬂatness due to the term (Eq.
(8)): sharper (higher curvature) minima should have higher , which makes it less likely to satisfy the condition.
The condition also depends on the amount of bias relative to gradient norm, i.e.,, higher relative bias makes it less likely to satisfy the condition for descent, which we also support through simulation (see Fig.
7C).
Taken together, variance and bias in gradient estimates promote gradient descent dynamics that converge to wide ﬂat minima, which can aid generalization.
This effect is empirically demonstrated by our ﬁrst example of VGG-16 networks trained on CIFAR-10 (Fig. 9).
3. E MPIRICAL VERIFICATION OF THE IMPACT OF VARIANCE AND BIAS ON GENERALIZATION
[Figure]
[Equation]
[Figure] [Equation] We follow the same experimental setting as described in Section 2.2 for verifying Theorems 3. and 3.2.
Owing to computational bottlenecks in loss Hessian estimation for high-dimensional nonlinear settings, we restrict our empirical validation to training linear networks for optimizing MSE loss.
In doing so, we gain full control over the loss Hessian via the input covariance statistics 4 — Fig.
7A & B demonstrate that our inferences from Theorem 3.1 hold empirically.
When the leading eigenvalue is less than the theoretical limit, a weight update step always leads to a decrease in L ; when the trailing eigenvalue is greater than the theoretical limit, a weight update step always leads to an increase in L .
Furthermore, the theoretical limit is higher for the case when there is no variance than with variance.
These observations demonstrate that following a noisy version of the gradient helps the network avoid sharp local minima.
Similarly, Fig.
7C demonstrates that our inferences from Theorem 3.2 hold empirically.
Speciﬁcally, when the ratio of bias to gradient norm is below the theoretical threshold, a weight update step always leads to a decrease in L .
D ISCUSSION
In both computational neuroscience and neuromorphic computing, gradient estimators are known to have variance and bias due to both algorithmic and hardware constraints (Laborieux et al., 2021;
M. Payvand et al., 2020a; Lillicrap et al., 2020; Pozzi et al., 2018; Sacramento et al., 2018;
Rubin et al., 2021; Roelfsema & Holtmaat, 2018; Bellec et al., 2020;
Neftci et al., 2019; Huh & Sejnowski, 2018; Zenke & Neftci, 2021).
Using mathematical analysis and empirical veriﬁcation, we found that the impact of variance and bias on learning is determined by the network size, activation function, gradient norm, and loss Hessian, such that the amount of variance and bias that are permissible or even desirable depends on these properties.
Though our empirical veriﬁcation was done using feedforward ANNs, our mathematical analysis was formulated to be as general as possible, and we believe that our assumptions are reasonable for most cases that computational neuroscientists and neuromorphic chip engineers face.
Furthermore, we add a corollary to Lemma 2.1 and discuss extensions of our analysis to multi-step optimization settings in Appendix A (see Corollary A.2).
Thus, our work can inform research in these areas by providing a guide for how much variance and bias relative to the gradient norm is reasonable for a given network.
4. R ELATED WORK
Our analysis was inspired in part by recent work by Raman et al.
(2019) characterizing bounds on learning performance in neural circuits.
They demonstrated that in the absence of noise in the gradient estimator, larger networks are always better at training, but with noise added, there is a size limit beyond which training is impaired.
Our work was also informed by research into generalization in deep ANNs, which provided the rationale for our analyses examining the potential for a learning algorithm to descend into sharp minima or not (Foret et al., 2020;
Chaudhari et al., 2019; Yao et al., 2018; Ghorbani et al., 2019; Smith et al., 2021).
As well, our work has some clear relationship to other work examining the variance of gradient estimators (e.g. Werfel et al.
, but here, we are asking a novel question, namely, how does a given amount of variance and bias impact performance for different network parameters?
More broadly, our work relates to the deep learning literature because modern ANNs rarely use the true loss gradient over the entire dataset to make weight updates.
Instead, it is common practice to add noise in different forms, e.g. SGD (Bottou, 2012), dropout (Srivastava et al., 2014), dropconnect (Wan et al., 2013) and label noise (Blanc et al., 2020; HaoChen et al., 2021; Damian et al., 2021).
But, the noise structure in these scenarios may differ from assumptions in our work.
For example,
SGD does not exhibit white noise structure (Xie et al., 2021; Zhu et al., 2019).
Likewise, dropout can be thought of as adding noise, but for a truly unbiased estimate of the gradient, one would have to consider all possible conﬁgurations of dropped neurons, something quite uncommon in practice.
4. L IMITATIONS AND FUTURE WORK
This work focused on characterizing the impact of variance and bias in gradient estimates, but it lacks any in-depth analysis of speciﬁc bio-plausible or neuromorphic learning algorithms.
Similarly, our work only characterizes learning performance, but does not provide a concrete proposal to mitigate excessive noise or bias in gradient approximations.
As such, our analyses can be used by other researchers to assess their learning algorithms but they do not directly speak to any speciﬁc learning algorithm nor provide mechanisms for improving existing algorithms.
Nonetheless, we believe that our work provides a framework to help develop such strategies, and we leave it to future work.
R EPRODUCIBILITY S TATEMENT
We strongly believe that reproducibility is critical for research progress in both machine learning and computational neuroscience.
To this end, we have provided thorough experimental details in the appendix, and in the supplementary materials, we have included the code to run all of the experiments and generate the ﬁgures.
Our code can also be accessed from the project’s github repo.
We believe that this information will allow the community to validate/replicate our results and further build on our work.
A CKNOWLEDGEMENTS
The authors would like to thank Abhishek Banerjee, Jonathan Cornford, and Shahab Bakhtiari for insightful discussions and helpful feedback over the course of the project, as well as Zahraa Chorghay and Colleen Gillon for their aesthetic contribution to the manuscript and ﬁgures.
This research was generously supported by Vanier Canada Graduate scholarship and Healthy Brains for Healthy Lives fellowship (A.G.); NSERC PGS-D and NSF AccelNet IN-BIC program, Grant No. OISEAM02 (Y.H.L.); NSERC Discovery Grant RGPIN-2018-04821, FRQS Research Scholar Award,
Junior 1 LAJGU0401-253188, Canada Research Chair in Neural Computations and Interfacing (G.L.); NSERC (Discovery Grant: RGPIN-2020-05105; Discovery Accelerator Supplement: RGPAS2020-00031), Healthy Brains, Healthy Lives (New Investigator Award: 2b-NISU-8) (B.A.R.); CIFAR
Learning in Machines and Brains Program (K.K. & B.A.R.); Canada CIFAR AI Chair program (G.L.
& B.A.R.).
This work was enabled by the material support of NVIDIA in the form of computational resources and support provided by Mila.