  We propose to address the issue of sample efficiency, in Deep Convolutional
Neural Networks (DCNN), with a semisupervised training strategy that combines
Hebbian learning with gradient descent: all internal layers (both convolutional
and fully connected) are pre-trained using an unsupervised approach based on
Hebbian learning, and the last fully connected layer (the classification layer)
is using Stochastic Gradient Descent (SGD). In fact, as Hebbian learning is an
unsupervised learning method, its potential lies in the possibility of training
the internal layers of a DCNN without labeled examples. Only the final fully
connected layer has to be trained with labeled examples. We performed
experiments on various object recognition datasets, in different regimes of
sample efficiency, comparing our semi-supervised (Hebbian for internal layers +
SGD for the final fully layer) approach with end-to-end supervised
backpropagation training. The results show that, in regimes where the number of
available labeled samples is low, our semi-supervised approach outperforms full
backpropagation in almost all the cases.
