Abstract-We present a spike-based unsupervised regenerative learning scheme to train Spiking Deep Networks (SpikeCNN) for object recognition problems using biologically realistic leaky integrate-and-fire neurons. The training methodology is based on the Auto-Encoder learning model wherein the hierarchical network is trained layer wise using the encoder-decoder principle. Regenerative learning uses spike-timing information and inherent latencies to update the weights and learn representative levels for each convolutional layer in an unsupervised manner. The features learnt from the final layer in the hierarchy are then fed to an output layer. The output layer is trained with supervision by showing a fraction of the labeled training dataset and performs the overall classification of the input. Our proposed methodology yields 0.95%/24.58% classification error on MNIST/CIFARIO datasets which is comparable with state-of-the-art results. The proposed methodology also introduces sparsity in the hierarchical feature representations on account of event-based coding result ing in computationally efficient learning. "Can machines think?" , the question brought up by Turing in his paper, [1], has led to the development of the field of brain-inspired machine learning wherein researchers have put substantial effort in building smarter, more aware devices and technology that have the potential of having human like understanding. In fact, large scale deep neural network architectures, such as Convolutional Neural Nets (CNNs), have demonstrated unprecedented performance (in terms of classification and recognition accuracy) in a wide range of computer vision and related applications. Such deep networks inspired by the cortical visual processing systems have seen increasing success in recent years due to the availability of more powerful computing hardware (GPU accelerators) and massive datasets for training. Regardless of their success, the substantial computational cost of training and testing such large-scale networks has limited their implementation to clouds and servers. In order to build devices with cognitive abilities, there is a need for specialized hardware with new computational theories. Spiking Neural Networks (SNNs) are a prime candidate for enabling such on-chip intelligence. Driven by brain-like asynchronous event based computations, SNNs focus their computational effort on currently active parts of the network, effectively saving power on the remaining part, thereby achieving orders of magnitude lesser power consump tion in comparison to their Artificial Neural Network (ANN) counterparts [2, 3] , which im plements a network of integrate-and-fire spiking neurons. However, TrueNorth does not incorporate any information pertaining to the learning mechanisms, which is at present a major constraint for realizing SNNs for real-world practical applications like visual and speech recognition among others. Thus, there is a need to develop efficient learning algorithms that might take the advantage of the specific features of SNNs (event-driven, low power, on-chip learning) while keeping the properties (general-purpose, scalable to larger problems with higher accuracy) of conventional ANN models. Recent efforts on training of deep spiking networks do not use spike-based learning rules, but instead start from a conventional ANN fully trained using labeled training data, followed by a conversion of the same into a model consisting of simple spiking neurons [5] [6] [7] . However, in order to extend the applicability of learning methods, the use of unlabeled data for machine learning is imperative. In the non-spiking domain, unsupervised learning of hierarchical regenerative models such as Auto-Encoders [8] have been successfully used to learn high-level features. The learnt features are then used as inputs to a supervised classification task [9] or to initialize a CNN [10] to avoid local minima. In this paper we develop upon Auto-Encoders where we build a spiking deep CNN by training each layer in the hierarchy in a purely unsupervised manner using the regenerative model and the temporal spike information to update the weights. SNNs are equipped with unsupervised weight-modification rules like Spike Timing Dependent Plas ticity (STDP) that learn the structure of input examples without using labels [11]. However, the network structure that has been successful in achieving competitive classification accuracy for pattern recognition problems is a single-layer SNN, which does not scale well to realistic sized high dimensional images in terms of computational complexity [12] . Moreover, STDP does not support learning hierarchical models that simulta neously represent multiple levels like edges or object parts in a visual context that is fundamental to a deep learning model. We propose regenerative learning using spike-timing information to implement layer-wise weight modification that learn representative levels for each convolutional layer. The features from the final layer of the deep convolutional spiking network (SpikeCNN) are then used for classification tasks. 299
A network inspired by the autoencoder of the neural networks literature is trained without labels layer-wise to reconstruct the MNIST and CIFAR-10 datasets, and whose output is trained in a supervised fashion to perform classification REF .
Unsupervised regenerative learning of hierarchical features in Spiking Deep Networks for object recognition
2016 International Joint Conference on Neural Networks (IJCNN)
2016 International Joint Conference on Neural Networks (IJCNN)
Computer Science