Biologically-Plausible Learning Algorithms Can Scale to Large Datasets

1 INTRODUCTION . Deep learning models today are highly successful in task performance , learning useful representations , and even matching representations in the brain ( Yamins et al. , 2014 ; Schrimpf et al. , 2018 ) . However , it remains a contentious issue whether these models reflect how the brain learns . Core to the problem is the fact that backpropagation , the learning algorithm underlying most of today ’ s deep networks , is difficult to implement in the brain given what we know about the brain ’ s hardware ( Crick 1989 ; however , see Hinton 2007 ) . One main reason why backpropagation seems implausible in the brain is that it requires sharing of feedforward and feedback weights . Since synapses are unidirectional in the brain , feedforward and feedback connections are physically distinct . Requiring them to shared their weights , even as weights are adjusted during learning , seems highly implausible . One approach to addressing this issue is to relax the requirement for weight-symmetry in error backpropagation . Surprisingly , when the feedback weights share only the sign but not the magnitude of the feedforward weights ( Liao et al. , 2016b ) or even when the feedback weights are random ( but fixed ) ( Lillicrap et al. , 2016 ) , they can still guide useful learning in the network , with performance comparable to and sometimes even better than performance of backpropagation , on datasets such as MNIST and CIFAR . Here , we refer to these two algorithms , respectively , as “ sign-symmetry ” and “ feedback alignment. ” Since weight symmetry in backpropagation is required for accurately propagating the derivative of the loss function through layers , the success of asymmetric feedback algorithms indicates that learning can be supported even by inaccurate estimation of the error derivative . In feedback alignment , the authors propose that the feedforward weights learn to align with the random feedback weights , thereby allowing feedback to provide approximate yet useful learning signals ( Lillicrap et al. , 2016 ) . However , a recent paper by Bartunov et al . ( 2018 ) finds that feedback alignment and a few other biologically-plausible algorithms , including variants of target propagation , do not generalize to larger and more difficult problems such as ImageNet ( Deng et al. , 2009 ) and perform much worse than backpropagation . Nevertheless , the specific conditions Bartunov et al . tested are somewhat restrictive . They only tested locally-connected networks ( i.e. , weight sharing is not allowed among convolution filters at different spatial locations ) , a choice that is motivated by biological plausibility but in practice limits the size of the network ( without weight sharing , each convolutional layer needs much more memory to store its weights ) , making it unclear whether poor performance was attributable solely to the algorithm , or to the algorithm on those architectures.1 Second , Bartunov et al . did not test sign-symmetry , which may be more powerful than feedback alignment since signsymmetric feedback weights may carry more information about the feedforward weights than the random feedback weights used in feedback alignment . In this work , we re-examine the performance of sign-symmetry and feedback alignment on ImageNet and MS COCO datasets using standard ConvNet architectures ( i.e. , ResNet-18 , AlexNet , and RetinaNet ) . We find that sign-symmetry can in fact train networks on both tasks , achieving similar performance to backpropagation on ImageNet and reasonable performance on MS COCO . In addition , we test the use of backpropagation exclusively in the last layer while otherwise using feedback alignment , hypothesizing that in the brain , the classifier layer may not be a fully-connected layer and may deliver the error signal through some other unspecified mechanism . Such partial feedback alignment can achieve better performance ( relative to backpropagation ) than in Bartunov et al . ( 2018 ) . Taken together , these results extend previous findings and indicate that existing biologicallyplausible learning algorithms remain viable options both for training artificial neural networks and for modeling how learning can occur in the brain . 2 METHODS . Consider a layer in a feedforward neural network . Let xi denote the input to the ith neuron in the layer and yj the output of the jth neuron . Let W denote the feedforward weight matrix and Wij the connection between input xi and output yj . Let f denote the activation function . Then , Equation 1 describes the computation in the feedforward step . Now , let B denote the feedback weight matrix and Bij the feedback connection between output yj and input xi , and let f ′ denote the derivative of the activation function f . Given the objective function E , the error gradient ∂E∂xi calculated in the feedback step is described by Equation 2. yj = f ( σj ) , σj = ∑ i Wijxi ( 1 ) ∂E ∂xi = ∑ j Bijf ′ ( σj ) ∂E ∂yj ( 2 ) Standard backpropagation requires B = W . Sign-symmetry ( Liao et al. , 2016b ) relaxes the above symmetry requirement by letting B = sign ( W ) , where sign ( · ) is the ( elementwise ) sign function . Feedback alignment ( Lillicrap et al. , 2016 ) uses a fixed random matrix as the feedback weight matrix B. Lillicrap et al . showed that through training , W is adjusted such that on average , eTWBe > 0 , where e is the error in the network ’ s output . This condition implies that the error correction signal Be lies within 90◦ of eTW , the error calculated by standard backpropagation . We implement both algorithms in PyTorch for convolutional and fully-connected layers and post the code at https : //github.com/willwx/sign-symmetry . 1Moreover , theoretical and experimental results of Poggio et al . ( 2017 ) suggest that weight-sharing is not the main reason for the good performance of ConvNets , at least when trained with backpropagation . 3 RESULTS . 3.1 SIGN-SYMMETRY PERFORMS WELL ON IMAGENET . TRAINING DETAILS We trained ResNet-18 ( He et al. , 2016 ) on ImageNet using 5 different training settings : 1 ) backpropagation ; 2 ) sign-symmetry for convolutional layers and backpropagation for the last , fully-connected layer ; 3 ) sign-symmetry for all ( convolutional and fully-connected ) layers ; 4 ) feedback alignment for convolutional layers and backpropagation for the fully-connected layer ; and 5 ) feedback alignment for all ( convolutional and fully-connected ) layers . In sign-symmetry , at each backward step , feedback weights were taken as the signs of the feedforward weights , scaled by the same scale λ used to initialize that layer.2 In feedback alignment , feedback weights were initialized once at the beginning as random variables from the same distribution used to initialize that layer . For backpropagation , standard training parameters were used ( SGD with learning rate 0.1 , momentum 0.9 , and weight decay 10−4 ) . For ResNet-18 with other learning algorithms , we used SGD with learning rate 0.053 , while momentum and weight decay remain unchanged . For AlexNet with all learning algorithms , standard training parameters were used ( SGD with learning rate 0.01 , momentum 0.9 , and weight decay 5 × 10−4 ) . We used a version of AlexNet ( Krizhevsky , 2014 , as used in torchvision ) which we slightly modified to add batch normalization ( Ioffe & Szegedy , 2015 ) before every nonlinearity and consequently removed dropout . For all experiments , we used a batch size of 256 , a learning rate decay of 10-fold every 10 epochs , and trained for 50 epochs . RESULTS In all cases , the network was able to learn ( Figure 1 , Table 1 ) . Remarkably , sign-symmetry only slightly underperformed backpropagation in this benchmark large dataset , despite the fact that signsymmetry does not accurately propagate either the magnitude or the sign of the error gradient . Hence , this result is not predicted by the performance of signSGD ( Bernstein et al. , 2018 ) , where weight updates use the sign of the gradients , but gradients are still calculate accurately ; or XNORNet ( Rastegari et al. , 2016 ) , where both feedforward and feedback weights are binary but symmetrical , so error backpropagation is still accurate . An intuitive explanation for this performance is that the skip-connections in ResNet help prevent the degradation of the gradient being passed through many layers of sign-symmetric feedback . However , sign-symmetry also performed similarly well 2For conv layers , λ= √ 2/ ( nkernel width · nkernel height · noutput channels ) ; for fully-connected layers , λ=1/ √ noutput . In this case , each entry in the feedback matrix has the same magnitude . We have also tested random fixed magnitudes and observe similar performance . 3Selected as the best from ( 0.1 , 0.05 , 0.01 , 0.001 , 0.0001 ) to backpropagation in a ( modified ) AlexNet architecture , which did not contain skip connections . Therefore , skip-connections alone do not explain the performance of sign-symmetry . In addition , although its performance was considerably worse , feedback alignment was still able to guide better learning in the network than reported by Bartunov et al . ( 2018 , their Figure 3 ) if we use backpropagation in the last layer . This condition is not unreasonable since , in the brain , the classifier layer is likely not a soft-max classifier and may deliver error signals by a different mechanism . We also tested using backpropagation exclusively for the last layer in a network otherwise trained with sign-symmetry , but the effect on the performance was minimal . One possibility why sign-symmetry performed better than feedback alignment is that in sign-symmetry , the feedback weight always tracks the sign of the feedforward weight , which may reduce the burden on the feedforward weight to learn to align with the feedback weight . Finally , in Liao et al . ( 2016b ) , Batch-Manhattan ( BM ) SGD was proposed as a way to stabilize training with asymmetric feedback algorithms . In our experience , standard SGD consistently worked better than BM for sign-symmetry , but BM may improve results for feedback alignment . We have not comprehensively characterized the effects of BM since many factors like learning rate can affect the outcome . Future experiments are needed to draw stronger conclusions . 3.2 MICROSOFT COCO DATASET . Besides the ImageNet classification task , we examined the performance of sign-symmetry on the MS COCO object detection task . Object detection is more complex than classification and might therefore require more complicated network architecture in order to achieve high accuracy . Thus , in this experiment we assessed the effectiveness of sign-symmetry in training networks that were more complicated and difficult to optimize . TRAINING DETAILS We trained the state-of-the-art object detection network RetinaNet proposed by Lin et al . ( 2018 ) on the COCO trainval35k split , which consists of 80k images from train and 35k random images from the 40k-image val set . RetinaNet comprises a ResNet-FPN backbone , a classification subnet , and a bounding box regressing subnet . The network was trained with three different training settings : 1 ) backpropagation for all layers ; 2 ) backpropagation for the last layer in both subnets and sign-symmetry for rest of the layers ; 3 ) backpropagation for the last layer in both subnets and feedback alignment for rest of the layers . We used a backbone ResNet-18 pretrained on ImageNet to initialize the network . In all the experiments , the network was trained with SGD with an initial learning rate of 0.01 , momentum of 0.9 , and weight decay of 0.0001 . We trained the network for 40k iterations with 8 images in each minibatch . The learning rate was divided by 10 at iteration 20k . RESULTS The results on COCO are similar to those on ImageNet , although the performance gap between SS and BP on COCO is slightly more prominent ( Figure 2 ) . A number of factors could have potentially contributed to this result . We followed the Feature Pyramid Network ( FPN ) architecture design choices , optimizers , and hyperparameters reported by Lin et al . ( 2018 ) ; these choices are all optimized for use with backpropagation instead of sign-symmetry . Hence , the results here represent a lowerbound on the performance of sign-symmetry for training networks on the COCO dataset .

Summary: This work adds to a growing literature on biologically plausible (BP) learning algorithms. Building off a study by Bartunov et al. that shows the deficiencies of some BP algorithms when scaled to difficult datasets, the authors evaluate a different algorithm, sign-symmetry, and conclude that there are indeed situations in which BP algorithms can scale. This seemingly runs counter to the conclusions of Bartunov et al.; while the authors state that their results are "complementary", they also state that the findings “directly conflict” with the results of Bartunov, concluding that BP algorithms remain viable options for both learning in artificial networks and the brain.