Credit Assignment Through Broadcasting a Global Error Vector

1 Introduction . Deep neural networks ( DNNs ) trained using backpropagation ( BP ) have achieved breakthroughs on a myriad of tasks [ 1 , 2 ] . The power of BP lies in its ability to discover intermediate feature representations by following the gradient of a loss function . In a manner analogous to DNN training , synapses in multilayered cortical circuits undergo plasticity that modulates neural activity several synapses downstream to improve performance on behavioral tasks [ 3 , 4 ] . However , neural circuits can not implement BP , implying that evolution has found another algorithm , or collection thereof [ 5 ] . This observation has motivated biologically plausible alternatives to BP [ 6 , 7 ] . Credit assignment algorithms based on broadcasting a global learning signal , such as node perturbation [ 8 ] , are attractive due to their biological plausibility , as the global signal could correspond to a neuromodulator that influences local synaptic plasticity [ 9 , 10 ] . However , perturbation methods are of little practical relevance as the variance in their gradient estimates is prohibitively large [ 7 , 11 ] . Instead , recent research has focused on methods that compute or estimate the gradient by transmitting detailed , unit-specific information to hidden layers through top-down feedback . The foremost example of such a method is BP . BP ’ s use of the feedforward weights in the feedback Code accompanying our paper is available at https : //github.com/davidclark1/VectorizedNets . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . backpropagation ( BP ) feedback alignment ( FA ) direct feedbackalignment ( DFA ) global error-vector broadcasting ( GEVB ) A B C D pathway , a property called weight symmetry , is not biologically plausible – this is the weight transport problem [ 12 ] . Another method of this class , feedback alignment ( FA ) , operates identically to BP but uses fixed , random matrices in the feedback pathway , thereby circumventing the weight transport problem [ 13 ] . FA inspired a further method called direct feedback alignment ( DFA ) , which delivers random projections of the output error vector directly to hidden units [ 14 , 15 ] . DFA approaches the performance of BP in certain modern architectures [ 16 ] , however its performance still lags behind that of BP in many cases [ 14 , 16–19 ] . A particularly striking shortcoming of DFA is its inability to train convolutional layers [ 16 , 19 , 20 ] . To be implemented in neural circuits , both FA and DFA require a biologically unrealistic “ error network ” to compute top-down learning signals , though models based on segregated dendrites aim to lift this requirement [ 21–23 ] . Here , we show that DNNs can be trained to perform on par with BP by broadcasting a single , global learning signal to all hidden units and applying local , Hebbian-like updates to the weights . Our proposed learning rule , called global error-vector broadcasting ( GEVB ) , is not perturbation-based , but instead distributes information about the output error throughout the network . Unlike DFA , which delivers a unique random projection of the output error vector to each hidden unit , GEVB broadcasts the same , unprojected error vector to all hidden units . Thus , GEVB involves no unit-specific feedback . This learning rule operates in a new class of DNNs called vectorized nonnegative networks ( VNNs ) , which have vector-valued units and nonnegative weights past the first layer . While vector-valued units are neuroscientifically speculative ( though see Section 7 for possible neural implementations ) , nonnegative weights are motivated by the fact that cortical projection neurons are excitatory [ 4 ] . The GEVB learning rule updates each weight by an amount proportional to the inner product of the presynaptic activation and the global error vector when the postsynaptic unit is active . Three-factor Hebbian plasticity encompasses synaptic update rules that depend on pre- and postsynaptic activity and a global neuromodulatory signal ( the third factor ) [ 24 , 25 ] . The GEVB learning rule is therefore a form of three-factor Hebbian plasticity with a vector-valued global third factor [ 26 ] . Our experimental results show that this form of global-error learning is surprisingly powerful , performing on par with BP in VNNs and overcoming DFA ’ s inability to train convolutional layers . 2 Credit assignment in conventional networks . Consider a fully connected network of L layers with pre-activations h ` i and activations a ` i , where i = 1 , . . . , n ` and ` = 0 , . . . , L. Let w ` ij denote the weights , b ` i the biases , and φ the pointwise nonlinearity . The forward dynamics are h ` i = n ` −1∑ j=1 w ` ija ` −1 j + b ` i , a ` i = φ ( h ` i ) , ( 1 ) with the understanding that φ is the identity function at the output layer L. The network inputs and outputs are a0i and a L i , respectively . We place a loss L on the output , and define ∂L/∂aLi = ei , where ei is the output error vector . On a single training example , a negative-gradient weight update can be written ∆w ` ij ∝ −δ ` iφ′ ( h ` i ) a ` −1j where δ ` i = ∂L ∂a ` i = nL∑ k=1 ∂aLk ∂a ` i ek . ( 2 ) BP , FA , and DFA are different methods of computing or approximating δ ` i . BP computes δ ` i exactly using the recurrence relation δ ` i = n ` +1∑ k=1 w ` +1ki φ ′ ( h ` +1k ) δ ` +1 k ( 3 ) with the initial condition δLi = ei ( Fig . 1A ) . FA approximates δ ` i by performing a backward pass identical to Eq . 3 , but using fixed , random feedback weights f ` ik in place of the transposed feedforward weights w ` ki ( Fig . 1B ) . This backward pass is given by δ ` i = n ` +1∑ k=1 f ` +1ik φ ′ ( h ` +1k ) δ ` +1 k ( 4 ) with the initial condition δLi = ei . Learning occurs under Eq . 4 due to learning-induced partial alignment of w ` ki with f ` ik [ 13 ] . Finally , DFA approximates δ ` i by setting the derivative ∂a L k /∂a ` i in the formula for δ ` i in Eq . 2 to a fixed , random matrix d ` ik ( Fig . 1C ) . This yields δ ` i = nL∑ k=1 d ` ikek . ( 5 ) In a similar manner to FA , learning occurs in DFA due to induced partial alignment of ∂aLk /∂a ` i with d ` ik [ 19 , 20 ] . The learning rule we propose further simplifies Eqns . 3 , 4 , and 5 by removing computation of the quantity analogous to δ ` i entirely ( Section 3.2 ) . A fundamental obstacle to accurate credit assignment in the absence of weight symmetry is that the sign of the gradient at each weight depends on detailed information about downstream weights to which the feedback pathway does not have access , even in the presence of partial alignment of feedforward and feedback weights . For example , suppose a DNN confidently predicts class c′ instead of the target class c. Then , the gradient at w ` ij depends on δ ` i ≈ ∂aLc′/∂a ` i − ∂aLc /∂a ` i , which can be positive or negative depending on a difference in the strengths of complex polysynaptic interactions between unit i in layer ` and units c and c′ in layer L. Intriguingly , this obstacle dissolves in networks with scalar output , positive weights past the first layer , and φ′ > 0 . In this case , we have δ ` i = ( ∂a L 1 /∂a ` i ) e1 with ∂a L 1 /∂a ` i > 0 . Thus , as per Eq . 2 , the gradient sign at w ` ij is equal to the sign of φ ′ ( h ` i ) a ` −1 j e1 . This expression has the form of a three-factor Hebbian rule . This simplification to credit assignment arising from sign-constrained weights in scalar-output networks was observed by both Balduzzi et al . [ 27 ] , who proposed the Kickback algorithm , and Lechner [ 18 ] , who used sign-constrained weights in DFA . However , the restriction to scalar-output ( e.g. , binary classification ) networks is highly limiting . The GEVB learning rule , described in the next section , generalizes this observation to vector-output ( e.g. , multi-way classification ) networks . 3 Credit assignment in vectorized nonnegative networks . 3.1 Vectorized nonnegative networks . Here , we introduce VNNs , the class of DNNs in which our proposed learning rule operates . Given a task with output dimension K , such as K-way classification , each VNN unit is a vector of dimension K. With the exception of the first hidden layer , connections obey “ vectorized weight sharing ” such that each vector unit computes a weighted linear combination of presynaptic vector units . Our notational convention is to use Latin letters to index different vector units and Greek letters to index the components of a single vector unit . Thus , Greek indices always run over 1 , . . . , K. Boldface variables represent all K components of a vector unit . For ` = 1 . . . , L , let h ` iµ denote the pre-activations and a ` iµ the activations . The inputs , a 0 i , are not vectorized and hence lack a µ subscript . The vector units in the first hidden layer compute a representation of the input using the weights w1iµj . For ` > 1 , connections obey vectorized weight sharing . Thus , these weights lack a µ subscript and are denoted by w ` ij . Crucially , these ` > 1 weights are nonnegative , consistent with excitatory cortical projection neurons . For ` = 1 , . . . , L , the biases are vector-valued and are denoted by b ` iµ . Finally , the vector-to-vector nonlinearity is denoted by Φ : RK → RK . This function mixes vector components in general . The forward dynamics are h ` iµ = n0∑ j=1 w1iµja 0 j + b 1 iµ ` = 1 n ` −1∑ j=1 w ` ija ` −1 jµ + b ` iµ ` > 1 a ` iµ = Φµ ( h ` i ) ( 6 ) with the understanding that Φ is the identity function at the output layer L. We assume that there is a single vector output unit aL1µ , denoted by a L µ for brevity , on which we place a loss L. We define ∂L/∂aLµ = eµ , where eµ is the output error vector . We will show that our proposed learning rule matches the sign of the gradient when the nonlinearity has the form Φµ ( h ) = G ( h ) hµ , ( 7 ) with G ( h ) ≥ 0 and piecewise constant . The function G induces a coupling between different vector components . While there are many options for this function , we choose G ( h ) = Θ ( t · h ) , ( 8 ) where Θ is the Heaviside step function and t is a gating vector that differs across units ( for brevity , we suppress its i and µ indices in our notation ) . We sample the gating vectors uniformly over { −1 , 1 } K at initialization and hold them constant throughout training and inference . In the K = 1 case , this choice of Φ reduces to a ReLU if t > 0 since Φ ( h ) = Θ ( th ) h = Θ ( h ) h = ReLU ( h ) . While we described the inputs as n0 scalar units with all-to-all connections to the first hidden layer , the inputs can be described equivalently as Kn0 vector units with vectorized weight-shared connections to the first hidden layer , unifying the ` = 1 and ` > 1 cases of the learning rule ( Appendix B ) . We use this convention henceforth .

Summary: This paper proposes to overcome an important biological implausibility of neural networks trained by backpropagation using two novelties: 1) to train neural networks using a new globally available error signal, 2) a new type of neural network with vector-valued units and nonnegative weights. Interestingly, the model shows that the proposed learning rules lead to update rules with the same sign as those obtained by gradient-descent.   This work justifies the biological plausibility of the learning rule by relating it to the growing literature on three-factor Hebbian rules and justifies the choice of nonnegative weights to observed cortical properties, as well as the presence of ON/OFF cells.   The performance of the proposed network is compared, on MNIST and CIFAR-10 datasets, against a very standard set network trained with BP, as well as more recent models trained using direct feedback alignment. The model outperforms competing DFA-trained models for fully connected and convolutional architectures but not for locally connected architectures.