  Recent work has shown that biologically plausible Hebbian learning can be
integrated with backpropagation learning (backprop), when training deep
convolutional neural networks. In particular, it has been shown that Hebbian
learning can be used for training the lower or the higher layers of a neural
network. For instance, Hebbian learning is effective for re-training the higher
layers of a pre-trained deep neural network, achieving comparable accuracy
w.r.t. SGD, while requiring fewer training epochs, suggesting potential
applications for transfer learning. In this paper we build on these results and
we further improve Hebbian learning in these settings, by using a nonlinear
Hebbian Principal Component Analysis (HPCA) learning rule, in place of the
Hebbian Winner Takes All (HWTA) strategy used in previous work. We test this
approach in the context of computer vision. In particular, the HPCA rule is
used to train Convolutional Neural Networks in order to extract relevant
features from the CIFAR-10 image dataset. The HPCA variant that we explore
further improves the previous results, motivating further interest towards
biologically plausible learning algorithms.
