Introduction
A longstanding question in neuroscience is how animals excel at learning complex behavior involving temporal dependencies across multiple timescales and thereafter generalize this learned behavior to unseen data. This requires solving the temporal credit assignment problem: how to assign the contribution of past neural states to future outcomes. To address this, neuroscientists are increasingly turning to the mathematical framework provided by recurrent neural networks (RNNs) to model learning mechanisms in the brain [ 1 , 2 ] . Temporal credit assignment in RNNs is typically achieved by backpropagation through time (BPTT), or other gradient descent-based optimization algorithms, none of which are biologically-plausible (or bio-plausible for short). Therefore, the use of RNNs as a framework to understand the computational principles of learning in the brain has motivated an influx of bio-plausible learning rules that approximate gradient descent [ 1 , 3 , 2 ] .

The performance of such rules is typically quantified by accuracy. Although the accuracy achieved by these rules is often comparable to true gradient descent, little is known about the breadth of the emergent solutions, namely how well they generalize. Broadly speaking, generalization refers to a trained model’s ability to adapt to previously unseen data, and is typically measured by the so-called generalization gap : the difference between training and testing error. This is especially important when learning complex tasks with nonlinear RNNs where the loss landscape is non-convex, and therefore, many solutions with comparable training accuracy can exist. These solutions, characterized as (local) minima in the loss landscape, can nonetheless exhibit drastically different levels of generalization (Figure 1 ). It is not clear if gradient-based methods like BPTT and the existing bio-plausible alternatives have a different tendency to converge to loss minima that provide better or worse generalization.

While the search for better predictors of the generalization performance remains an open issue in deep learning research [ 4 ] , recent extensive studies identify flatness of the loss landscape at the solution point as a promising predictor for generalization [ 5 , 6 , 7 , 8 , 9 ] . Leveraging these empirical and theoretical findings, we ask : how do proposed biologically-motivated gradient approximations affect the flatness of the converged solution on loss landscape, and thereby, generalization?

Our overarching goal is to investigate generalization trends for existing bio-plausible temporal credit assignment rules, and in particular, examine how truncation-based bio-plausible gradient approximations can affect such trends. Specifically, our contributions are summarized as follows:

In numerical experiments, we demonstrate across several well-known neuroscience and machine learning benchmark tasks that state-of-the-art (SoTA) bio-plausible learning rules for training RNNs exhibit worse and more variable generalization gaps, compared to true (stochastic) gradient descent (Figure 2 A-C).

Using the same experiments, we show that bio-plausible learning rules tend to approach high-curvature regions in synaptic weight space as measured by the loss’ Hessian eigenspectrum (Figure 2 D-F). Further, we verify that this correlates with worse generalization (Figure 2 D-F and 3 ), which is consistent with the literature.

We present a theorem to explain this phenomenon by examining the weight update equation as a discrete dynamical system, which sheds light on a potential connection between gradient alignment and the preference over converging to narrower minima (Theorem 1 , Figure 4 ).

Given the core components in designing artificial neural networks: data, objective functions, learning rules and architectures [ 10 ] , we investigate different learning rules while holding the data, objective function and architecture constant. SoTA RNN learning rules investigated include a three-factor rule with symmetric feedback (symmetric e-prop [ 11 ] ), a three-factor rule using random feedback weights (RFLO [ 12 ] ) and a multi-factor rule using local modulatory signaling (MDGL) [ 13 ] . For an in-depth explanation of how these rules are implemented and why they are bio-plausible, please refer to Appendix A.2 . We also encourage the reader to visit Appendix B for Theorem 1 proof and discussion on loss landscape geometry. In the last paragraph of the Discussion section, we discuss potential remedies implemented by the brain and provide preliminary results (Appendix Figure 5 ). To our knowledge, our analysis is the first to highlight and quantitatively provide a mechanistic explanation of the reason for this gap in solution quality between artificial and bio-plausible learning rules for RNNs, thereby motivating further investigations into how the brain learns solutions that generalize.