Convergence and Alignment of Gradient Descent with Random Backpropagation Weights

Stochastic gradient descent with backpropagation is the workhorse of artificial neural networks . It has long been recognized that backpropagation fails to be a biologically plausible algorithm . Fundamentally , it is a non-local procedure— updating one neuron ’ s synaptic weights requires knowledge of synaptic weights or receptive fields of downstream neurons . This limits the use of artificial neural networks as a tool for understanding the biological principles of information processing in the brain . Lillicrap et al . ( 2016 ) propose a more biologically plausible “ feedback alignment ” algorithm that uses random and fixed backpropagation weights , and show promising simulations . In this paper we study the mathematical properties of the feedback alignment procedure by analyzing convergence and alignment for two-layer networks under squared error loss . In the overparameterized setting , we prove that the error converges to zero exponentially fast , and also that regularization is necessary in order for the parameters to become aligned with the random backpropagation weights . Simulations are given that are consistent with this analysis and suggest further generalizations . These results contribute to our understanding of how biologically plausible algorithms might carry out weight learning in a manner different from Hebbian learning , with performance that is comparable with the full non-local backpropagation algorithm . 1 Introduction . The roots of artificial neural networks draw inspiration from networks of biological neurons ( Rumelhart et al. , 1986a ; Elman et al. , 1996 ; Medler , 1998 ) . Grounded in simple abstractions of membrane potentials and firing , neural networks are increasingly being employed as a computational tool for better understanding the biological principles of information processing in the brain ; examples include Yildirim et al . ( 2019 ) and Yamins & DiCarlo ( 2016 ) . Even when full biological fidelity is not required , it can be useful to better align the computational abstraction with neuroscience principles . Stochastic gradient descent has been a workhorse of artificial neural networks . Conveniently , calculation of gradients can be carried out using the backpropagation algorithm , where reverse mode automatic differentiation provides a powerful way of computing the derivatives for general architectures ( Rumelhart et al. , 1986b ) . Yet it has long been recognized that backpropagation fails to be a biologically plausible algorithm . Fundamentally , it is a non-local procedure—updating the weight between a presynaptic and postsynaptic neuron requires knowledge of the weights between the postsynaptic neuron and other neurons . No known biological mechanism exists for propagating 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . information in this manner . This limits the use of artificial neural networks as a tool for understanding learning in the brain . A wide range of approaches have been explored as a potential basis for learning and synaptic plasticity . Hebbian learning is the most fundamental procedure for adjusting weights , where repeated stimulation by a presynaptic neuron that results in the subsequent firing of the postsynapic neuron will result in an increased strength in the connection between the two cells ( Hebb , 1961 ; Paulsen & Sejnowski , 2000 ) . Several variants of Hebbian learning , some making connections to principal components analysis , have been proposed ( Oja , 1982 ; Sejnowski & Tesauro , 1989 ; Sejnowski , 1999 ) . In this paper , our focus is on a formulation of Lillicrap et al . ( 2016 ) based on random backpropagation weights that are fixed during the learning process , called the “ feedback alignment ” ( FA ) algorithm . Lillicrap et al . ( 2016 ) show that the model can still learn from data , and observe the interesting phenomenon that the error signals propagated with the forward weights align with those propagated with fixed random backward weights during training . Direct feedback alignment ( DFA ) ( Nøkland , 2016 ) extends FA by adding skip connections to send the error signals directly to each hidden layer , allowing parallelization of weight updates . Empirical studies given by Launay et al . ( 2020 ) show that DFA can be successfully applied to train a number of modern deep learning models , including transformers . Based on DFA , Frenkel et al . ( 2021 ) proposes direct the random target projection ( DRTP ) algorithm that trains the network weights with a random projection of the target vector instead of the error , and shows alignment for linear networks . Related proposals , including methods based on the use of differences of neuron activities , have been made in a series of recent papers ( Akrout et al. , 2019 ; Bellec et al. , 2019 ; Lillicrap et al. , 2020 ) . A comparison of some of these methods is made by Bartunov et al . ( 2018 ) . The use of random feedback weights , which are not directly tied to the forward weights , removes issues of non-locality . However , it is not clear under what conditions optimization of error and learning can be successful . While Lillicrap et al . ( 2016 ) give suggestive simulations and some analysis for the linear case , it has been an open problem to explain the behavior of this algorithm for training the weights of a neural network . In this paper , we study the mathematical properties of the feedback alignment procedure by analyzing convergence and alignment for two-layer networks under squared error loss . In the overparameterized setting , we prove that the error converges to zero exponentially fast . We also show , unexpectedly , that the parameters become aligned with the random backpropagation weights only when regularization is used . Simulations are given that are consistent with this analysis and suggest further generalizations . The following section gives further background and an overview of our results . 2 Problem Statement and Overview of Results . In this section we provide a formulation of the backpropagation algorithm to establish notation and the context for our analysis . We then formulate the feedback aligment algorithm that uses random backpropation weights . A high-level overview of our results is then presented , together with some of the intuition and proof techniques behind these results ; we also contrast with what was known previously . We mainly consider two-layer neural networks in the regression setting , specified by a family of functions f : Rd → R with input dimension d , sample size n , and p neurons in the hidden layer . For an input x ∈ Rd , the network outputs f ( x ) = 1√ p p∑ r=1 βrψ ( wᵀrx ) = 1 √ p β ᵀ ψ ( Wx ) , ( 2.1 ) where W = ( w1 , ... , wp ) ᵀ ∈ Rp×d and β = ( β1 , ... , βp ) ᵀ ∈ Rp represent the feed-forward weights in the first and second layers , and ψ denotes an element-wise activation function . The scaling by √ p is simply for convenience in the analysis . Given n input-response pairs { ( xi , yi ) } ni=1 , the training objective is to minimize the squared error L ( W , β ) = 12 n∑ i=1 ( yi − f ( xi ) ) 2 . ( 2.2 ) Standard gradient descent attempts to minimize ( 2.2 ) by updating the feed-forward weights following gradient directions according to βr ( t+ 1 ) = βr ( t ) − η ∂L ∂βr ( W ( t ) , β ( t ) ) wr ( t+ 1 ) = wr ( t ) − η ∂L ∂wr ( W ( t ) , β ( t ) ) , for each r ∈ [ p ] , where η > 0 denotes the step size . We initialize β ( 0 ) and wr ( 0 ) as standard Gaussian vectors . We introduce the notation f ( t ) , e ( t ) ∈ Rn , with fi ( t ) = f ( xi ) denoting the network output on input xi when the weights are W ( t ) and β ( t ) , and ei ( t ) = yi − fi ( t ) denoting the corresponding prediction error or residual . With this notation , the gradients are expressed as ∂L ∂βr = 1√ p n∑ i=1 eiψ ( wᵀrxi ) , ∂L ∂wr = 1√ p n∑ i=1 eiβrψ ′ ( wᵀrxi ) xi . Here it is seen that the the gradient of the first-layer weights ∂L∂wr involves not only the local input xi and the change in the response of the r-th neuron , but also the backpropagated error signal eiβr . The appearance of βr is , of course , due to the chain rule ; but in effect it requires that the forward weights between layers are identical to the backward weights under error propagation . There is no evidence of biological mechanisms that would enable such “ synaptic symmetry. ” In the feedback alignment procedure of ( Lillicrap et al. , 2016 ) , when updating the weightswr , the error signal is weighted , and propagated backward , not by the second layer feedforward weights β , but rather by a random set of weights b ∈ Rp that are fixed during the course of training . Equivalently , the gradients for the first layer are replaced by the terms ∂̃L ∂wr = 1√ p n∑ i=1 eibrψ ′ ( wᵀrxi ) xi . ( 2.3 ) Note , however , that this update rule does not correspond to the gradient with respect to a modified loss function . The use of a random weight br when updating the first layer weights wr does not violate locality , and could conceivably be implemented by biological mechanisms ; we refer to Lillicrap et al . ( 2016 ) ; Bartunov et al . ( 2018 ) ; Lillicrap et al . ( 2020 ) for further discussion . A schematic of the relationship between the two algorithms is shown in Figure 1 . We can now summarize the main results and contributions of this paper . Our first result shows that the error converges to zero when using random backpropagation weights . • Under Gaussian initialization of the parameters , if the model is sufficiently overparameterized with p n , then the error converges to zero linearly . Moreover , the parameters satisfy ‖wr ( t ) − wr ( 0 ) ‖ = Õ ( n√ p ) and |βr ( t ) − βr ( 0 ) | = Õ ( n√ p ) . The precise assumptions and statement of this result are given in Theorem 3.2 . The proof shows in the over-parameterized regime that the weights only change by a small amount . While related to results for standard gradient descent , new methods are required because the “ effective kernel ” is not positive semi-definite . We next turn to the issue of alignment of the second layer parameters β with the random backpropagation weights b . Such alignment was first observed in the original simulations of Lillicrap et al . ( 2016 ) . With h ∈ Rp denoting the hidden layer of the two-layer network , the term δBP ( h ) : = ∂L∂h = 1√ pβ ∑n i=1 ei represents how the error signals ei are sent backward to update the feed-forward weights . With the use of random backpropagation weights , the error is instead propagated backward as δFA ( h ) = 1√pb ∑n i=1 ei . Lillicrap et al . ( 2016 ) notice a decreasing angle between δBP ( h ) and δFA ( h ) during training , which is a sufficient condition to ensure that the algorithm converges . In the case of k-way classification , the last layer has k nodes , β and b are p × k matrices , and each error term ei is a k-vector . In the regression setting , k = 1 so the angle between δBP ( h ) and δFA ( h ) is the same as the angle between β and b . Intuitively , the possibility for alignment is seen in the fact that while the updates for W use the error weighted by the random weights b , the updates for β indirectly involveW , allowing for the possibility that dependence on b will be introduced into β . Our first result shows that , in fact , alignment will not occur in the over-parameterized setting . ( So , while the error may still converge , “ feedback alignment ” may be a bit of a misnomer for the algorithm . ) • The cosine of the angle between the p-dimensional vectors δFA and δBP satisfies cos∠ ( δFA , δBP ( t ) ) = cos∠ ( b , β ( t ) ) = O ( n√ p ) . However , we show that regularizing the parameters will cause δBP to align with δFA and therefore the parameters β to align with b . Since β ( 0 ) and b are high dimensional Gaussian vectors , they are nearly orthogonal with high probability . The effect of regularization can be seen as shrinking the component of β ( 0 ) in the parameters over time . Our next result establishes this precisely in the linear case . • Supposing that ψ ( u ) = u , then introducing a ridge penalty λ ( t ) ‖β‖2 where λ ( t ) = λ for t ≤ T and λ ( t ) = 0 for t > T on β causes the parameters to align , with cos∠ ( b , β ( t ) ) ≥ c > 0 for sufficiently large t. The technical conditions are given in Theorem 4.6 . Our simulations are consistent with this result , and also show alignment with a constant regularization λ ( t ) ≡ λ , for both linear and nonlinear activation functions . Finally , we complement this result by showing that convergence is preserved with regularization , for general activation functions . This is presented in Theorem 4.2 .

Summary: This paper builds upon the work of Lillicrap et al 2016 of attempting to find biologically plausible methods for implementing backpropagation. They thoroughly describe the issues with non-local information and why there are limits to using backprop-based learning to understand more about biological neural systems.  They describe the algorithm of feedback alignment using random back-propagated weights.  The authors go on to confirm some of the results of the Lillicrap paper, and find a surprising and novel result that regularization is required to produce convergence in over-parameterized settings. The experimental section of the paper shows some work on a more standard ML benchmark task of MNIST classification.  They show the results that regularization is required to get good performance on the task.