B.3 Discussion on generality of Theorem 1
The proof above examines a special case where the Jacobian of weight update equations becomes rank-1. We remind the reader that for the case of multivariate loss, higher rank cases or batch updates, we would not have arrived at the rank-1 Jacobian step (Eq. 30 ). For many tasks considered in neuroscience, the rank 1 case can apply, which explains the validity of Theorem 1 . We also note in passing that for the case of stochastic gradient descent, the Hessian Jacobian correspondence (Eq. 23 ) would point to loss’ Hessian matrix evaluated on a single example, which could reflect the robustness against perturbing that particular example. Moreover, simulation results show our conclusion holds in higher rank cases (Figure 4 ). The challenge of generalizing the proof to higher Jacobian rank case is that we are no longer guaranteed that the leading eigenvectors of BPTT Jacobian coincide with the leading eigenvectors of an approximate rule Jacobian. Thus, it becomes much harder to relate |\widehat{\lambda^{J}_{1}}| and |\lambda^{J}_{1}| . Rather than providing further proof, we provide an intuition for why our conclusion — where the convergence behavior between rules differs by their step length along the gradient direction — can hold in higher rank cases under Assumption 1 .

Approximation error vector \vec{e} (but not \vec{g} ) lies orthogonal to the subspace spanned by the leading Hessian eigenvectors. Here, leading Hessian eigenvectors refer to the eigenvectors corresponding to the outlier Hessian eigenvalues in light of the well-known observation that there exists only a few large (outlier) eigenvalues and the rest are near zero [ 174 , 117 , 118 ] ).

The ramification of Assumption 1 is that \vec{e} will lie in the subspace spanned by eigenvectors corresponding to tiny eigenvalues, making H\vec{e} tiny. In the extreme scenario where \vec{e} lies in the null space of H , H\vec{e} would be 0 . We verify this assumption numerically in Figure 10 . We saw from the proof above that this assumption is automatically satisfied in the rank-1 Jacobian case. We remark that this assumption should not hold for stochastic gradient noise (SGN), as the SGN covariance matrix is well aligned with the Hessian matrix near a minima [ 123 ] . This could be why \vec{e} , unlike stochastic gradient noise, does not seem to be contributing much to escaping narrow minima.

We consider the case of small enough weight updates such that the loss surface can be approximated using second-order Taylor expansion. Thus, the loss change after one update becomes:

\Delta L \approx\Delta W^{T}\vec{g}+\frac{1}{2}\Delta W^{T}H\Delta W (38) =-\eta_{B}\vec{g}^{T}\vec{g}+\frac{1}{2}\eta_{B}^{2}\vec{g}^{T}H\vec{g},\quad(for exact rule) (39) \widehat{\Delta L} \approx\widehat{\Delta W}^{T}\vec{g}+\frac{1}{2}\widehat{\Delta W}^{T}H\widehat{\Delta W} (40) =-\eta_{e}\vec{\hat{g}}^{T}\vec{g}+\frac{1}{2}\eta_{e}^{2}\vec{\hat{g}}^{T}H\vec{\hat{g}}.\quad(for an approximate rule) (41)

We next focus on the first- and second-order Taylor terms ( T_{1} and T_{2} ) for the exact rule as well as the terms ( \widehat{T_{1}} and \widehat{T_{2}} ) for an approximate rule:

T_{1}:=\eta_{B}\vec{g}^{T}\vec{g},\widehat{T_{1}}:=\eta_{e}\vec{\hat{g}}^{T}\vec{g},T_{2}:=\frac{1}{2}\eta_{B}^{2}\vec{g}^{T}H\vec{g},\widehat{T_{2}}:=\frac{1}{2}\eta_{B}^{2}\vec{\hat{g}}^{T}H\vec{\hat{g}},

and we note that first and second Taylor terms can determine how likely the update will be trapped in a local minimum:

\Delta L <0 (enables descend)\quad\rightarrow T_{1}>T_{2} (42) \Delta L >0 (restricts convergence)\quad\rightarrow T_{1}<T_{2} (43) \widehat{\Delta L} <0 (enables descend)\quad\rightarrow\widehat{T_{1}}>\widehat{T_{2}} (44) \widehat{\Delta L} >0 (restricts convergence)\quad\rightarrow\widehat{T_{1}}<\widehat{T_{2}}

Given their central role in determining convergence, we compare these terms between exact gradient descent learning and an approximate rule. For the first Taylor term (T1), it is easy to see that:

\vec{\hat{g}}^{T}\vec{g}=\rho\vec{g}^{T}\vec{g}.

For the second Taylor term (T2) and if H is symmetric:

\vec{\hat{g}}^{T}H\vec{\hat{g}} =\rho^{2}\vec{g}^{T}H\vec{g}+2\rho\vec{g}^{T}{\underbrace{\textstyle H\vec{e}}_{\mathclap{ $\approx 0$ }}}+\rho^{2}\vec{e}^{T}{\underbrace{\textstyle H\vec{e}}_{\mathclap{$\approx 0$, Assumption~{\ref{assume:orth}}}}} (46) \approx\rho^{2}\vec{g}^{T}H\vec{g}

To match the convergence behavior between exact gradient descent and an approximate rule (Eq. 45 ) on a (locally) second-order loss surface, we can make (T_{1},T_{2}) approximately equal to (\widehat{T_{1}},\widehat{T_{2}}) by setting \eta_{B}=\rho\eta_{e} , which predicts our numerical results (Figure 4 ). We note that if Assumption 1 were not satisfied, then the above might not hold. We note in passing that if we can satisfy Assumption 1 without being near an optimum, then we may not need the negligible training error assumption.