Convergence and Alignment of Gradient Descent with Random Backpropagation Weights

Stochastic gradient descent with backpropagation is the workhorse of artificial neural networks . It has long been recognized that backpropagation fails to be a biologically plausible algorithm . Fundamentally , it is a non-local procedure— updating one neuron ’ s synaptic weights requires knowledge of synaptic weights or receptive fields of downstream neurons . This limits the use of artificial neural networks as a tool for understanding the biological principles of information processing in the brain . Lillicrap et al . ( 2016 ) propose a more biologically plausible “ feedback alignment ” algorithm that uses random and fixed backpropagation weights , and show promising simulations . In this paper we study the mathematical properties of the feedback alignment procedure by analyzing convergence and alignment for two-layer networks under squared error loss . In the overparameterized setting , we prove that the error converges to zero exponentially fast , and also that regularization is necessary in order for the parameters to become aligned with the random backpropagation weights . Simulations are given that are consistent with this analysis and suggest further generalizations . These results contribute to our understanding of how biologically plausible algorithms might carry out weight learning in a manner different from Hebbian learning , with performance that is comparable with the full non-local backpropagation algorithm . 1 Introduction . The roots of artificial neural networks draw inspiration from networks of biological neurons ( Rumelhart et al. , 1986a ; Elman et al. , 1996 ; Medler , 1998 ) . Grounded in simple abstractions of membrane potentials and firing , neural networks are increasingly being employed as a computational tool for better understanding the biological principles of information processing in the brain ; examples include Yildirim et al . ( 2019 ) and Yamins & DiCarlo ( 2016 ) . Even when full biological fidelity is not required , it can be useful to better align the computational abstraction with neuroscience principles . Stochastic gradient descent has been a workhorse of artificial neural networks . Conveniently , calculation of gradients can be carried out using the backpropagation algorithm , where reverse mode automatic differentiation provides a powerful way of computing the derivatives for general architectures ( Rumelhart et al. , 1986b ) . Yet it has long been recognized that backpropagation fails to be a biologically plausible algorithm . Fundamentally , it is a non-local procedure—updating the weight between a presynaptic and postsynaptic neuron requires knowledge of the weights between the postsynaptic neuron and other neurons . No known biological mechanism exists for propagating 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . information in this manner . This limits the use of artificial neural networks as a tool for understanding learning in the brain . A wide range of approaches have been explored as a potential basis for learning and synaptic plasticity . Hebbian learning is the most fundamental procedure for adjusting weights , where repeated stimulation by a presynaptic neuron that results in the subsequent firing of the postsynapic neuron will result in an increased strength in the connection between the two cells ( Hebb , 1961 ; Paulsen & Sejnowski , 2000 ) . Several variants of Hebbian learning , some making connections to principal components analysis , have been proposed ( Oja , 1982 ; Sejnowski & Tesauro , 1989 ; Sejnowski , 1999 ) . In this paper , our focus is on a formulation of Lillicrap et al . ( 2016 ) based on random backpropagation weights that are fixed during the learning process , called the “ feedback alignment ” ( FA ) algorithm . Lillicrap et al . ( 2016 ) show that the model can still learn from data , and observe the interesting phenomenon that the error signals propagated with the forward weights align with those propagated with fixed random backward weights during training . Direct feedback alignment ( DFA ) ( Nøkland , 2016 ) extends FA by adding skip connections to send the error signals directly to each hidden layer , allowing parallelization of weight updates . Empirical studies given by Launay et al . ( 2020 ) show that DFA can be successfully applied to train a number of modern deep learning models , including transformers . Based on DFA , Frenkel et al . ( 2021 ) proposes direct the random target projection ( DRTP ) algorithm that trains the network weights with a random projection of the target vector instead of the error , and shows alignment for linear networks . Related proposals , including methods based on the use of differences of neuron activities , have been made in a series of recent papers ( Akrout et al. , 2019 ; Bellec et al. , 2019 ; Lillicrap et al. , 2020 ) . A comparison of some of these methods is made by Bartunov et al . ( 2018 ) . The use of random feedback weights , which are not directly tied to the forward weights , removes issues of non-locality . However , it is not clear under what conditions optimization of error and learning can be successful . While Lillicrap et al . ( 2016 ) give suggestive simulations and some analysis for the linear case , it has been an open problem to explain the behavior of this algorithm for training the weights of a neural network . In this paper , we study the mathematical properties of the feedback alignment procedure by analyzing convergence and alignment for two-layer networks under squared error loss . In the overparameterized setting , we prove that the error converges to zero exponentially fast . We also show , unexpectedly , that the parameters become aligned with the random backpropagation weights only when regularization is used . Simulations are given that are consistent with this analysis and suggest further generalizations . The following section gives further background and an overview of our results . 2 Problem Statement and Overview of Results . In this section we provide a formulation of the backpropagation algorithm to establish notation and the context for our analysis . We then formulate the feedback aligment algorithm that uses random backpropation weights . A high-level overview of our results is then presented , together with some of the intuition and proof techniques behind these results ; we also contrast with what was known previously . We mainly consider two-layer neural networks in the regression setting , specified by a family of functions f : Rd → R with input dimension d , sample size n , and p neurons in the hidden layer . For an input x ∈ Rd , the network outputs f ( x ) = 1√ p p∑ r=1 βrψ ( wᵀrx ) = 1 √ p β ᵀ ψ ( Wx ) , ( 2.1 ) where W = ( w1 , ... , wp ) ᵀ ∈ Rp×d and β = ( β1 , ... , βp ) ᵀ ∈ Rp represent the feed-forward weights in the first and second layers , and ψ denotes an element-wise activation function . The scaling by √ p is simply for convenience in the analysis . Given n input-response pairs { ( xi , yi ) } ni=1 , the training objective is to minimize the squared error L ( W , β ) = 12 n∑ i=1 ( yi − f ( xi ) ) 2 . ( 2.2 ) Standard gradient descent attempts to minimize ( 2.2 ) by updating the feed-forward weights following gradient directions according to βr ( t+ 1 ) = βr ( t ) − η ∂L ∂βr ( W ( t ) , β ( t ) ) wr ( t+ 1 ) = wr ( t ) − η ∂L ∂wr ( W ( t ) , β ( t ) ) , for each r ∈ [ p ] , where η > 0 denotes the step size . We initialize β ( 0 ) and wr ( 0 ) as standard Gaussian vectors . We introduce the notation f ( t ) , e ( t ) ∈ Rn , with fi ( t ) = f ( xi ) denoting the network output on input xi when the weights are W ( t ) and β ( t ) , and ei ( t ) = yi − fi ( t ) denoting the corresponding prediction error or residual . With this notation , the gradients are expressed as ∂L ∂βr = 1√ p n∑ i=1 eiψ ( wᵀrxi ) , ∂L ∂wr = 1√ p n∑ i=1 eiβrψ ′ ( wᵀrxi ) xi . Here it is seen that the the gradient of the first-layer weights ∂L∂wr involves not only the local input xi and the change in the response of the r-th neuron , but also the backpropagated error signal eiβr . The appearance of βr is , of course , due to the chain rule ; but in effect it requires that the forward weights between layers are identical to the backward weights under error propagation . There is no evidence of biological mechanisms that would enable such “ synaptic symmetry. ” In the feedback alignment procedure of ( Lillicrap et al. , 2016 ) , when updating the weightswr , the error signal is weighted , and propagated backward , not by the second layer feedforward weights β , but rather by a random set of weights b ∈ Rp that are fixed during the course of training . Equivalently , the gradients for the first layer are replaced by the terms ∂̃L ∂wr = 1√ p n∑ i=1 eibrψ ′ ( wᵀrxi ) xi . ( 2.3 ) Note , however , that this update rule does not correspond to the gradient with respect to a modified loss function . The use of a random weight br when updating the first layer weights wr does not violate locality , and could conceivably be implemented by biological mechanisms ; we refer to Lillicrap et al . ( 2016 ) ; Bartunov et al . ( 2018 ) ; Lillicrap et al . ( 2020 ) for further discussion . A schematic of the relationship between the two algorithms is shown in Figure 1 . We can now summarize the main results and contributions of this paper . Our first result shows that the error converges to zero when using random backpropagation weights . • Under Gaussian initialization of the parameters , if the model is sufficiently overparameterized with p n , then the error converges to zero linearly . Moreover , the parameters satisfy ‖wr ( t ) − wr ( 0 ) ‖ = Õ ( n√ p ) and |βr ( t ) − βr ( 0 ) | = Õ ( n√ p ) . The precise assumptions and statement of this result are given in Theorem 3.2 . The proof shows in the over-parameterized regime that the weights only change by a small amount . While related to results for standard gradient descent , new methods are required because the “ effective kernel ” is not positive semi-definite . We next turn to the issue of alignment of the second layer parameters β with the random backpropagation weights b . Such alignment was first observed in the original simulations of Lillicrap et al . ( 2016 ) . With h ∈ Rp denoting the hidden layer of the two-layer network , the term δBP ( h ) : = ∂L∂h = 1√ pβ ∑n i=1 ei represents how the error signals ei are sent backward to update the feed-forward weights . With the use of random backpropagation weights , the error is instead propagated backward as δFA ( h ) = 1√pb ∑n i=1 ei . Lillicrap et al . ( 2016 ) notice a decreasing angle between δBP ( h ) and δFA ( h ) during training , which is a sufficient condition to ensure that the algorithm converges . In the case of k-way classification , the last layer has k nodes , β and b are p × k matrices , and each error term ei is a k-vector . In the regression setting , k = 1 so the angle between δBP ( h ) and δFA ( h ) is the same as the angle between β and b . Intuitively , the possibility for alignment is seen in the fact that while the updates for W use the error weighted by the random weights b , the updates for β indirectly involveW , allowing for the possibility that dependence on b will be introduced into β . Our first result shows that , in fact , alignment will not occur in the over-parameterized setting . ( So , while the error may still converge , “ feedback alignment ” may be a bit of a misnomer for the algorithm . ) • The cosine of the angle between the p-dimensional vectors δFA and δBP satisfies cos∠ ( δFA , δBP ( t ) ) = cos∠ ( b , β ( t ) ) = O ( n√ p ) . However , we show that regularizing the parameters will cause δBP to align with δFA and therefore the parameters β to align with b . Since β ( 0 ) and b are high dimensional Gaussian vectors , they are nearly orthogonal with high probability . The effect of regularization can be seen as shrinking the component of β ( 0 ) in the parameters over time . Our next result establishes this precisely in the linear case . • Supposing that ψ ( u ) = u , then introducing a ridge penalty λ ( t ) ‖β‖2 where λ ( t ) = λ for t ≤ T and λ ( t ) = 0 for t > T on β causes the parameters to align , with cos∠ ( b , β ( t ) ) ≥ c > 0 for sufficiently large t. The technical conditions are given in Theorem 4.6 . Our simulations are consistent with this result , and also show alignment with a constant regularization λ ( t ) ≡ λ , for both linear and nonlinear activation functions . Finally , we complement this result by showing that convergence is preserved with regularization , for general activation functions . This is presented in Theorem 4.2 .

Summary: This paper studies the convergence of Feedback Alignment, and proves that in the over-parameterized setting the error converges to zero exponentially fast. This is made possible by taking inspirations from recent work on the NTK, albeit with specific challenges as some quantities are not obviously positive semi-definite. Furthermore, it makes the somewhat surprising finding that regularization helps alignment, proving this in linear networks, and confirming this finding in simulations on Gaussian data and MNIST.