EE 660 Machine Learning from Signals : Foundations and Methods Project Title : Analysis of Unsupervised Feature Learning Methods for Image Classification

This project investigates the utility of the unsupervised feature learning algorithm for the task of image classification. The state of the art methods for image classification use a pre-learning stage with deep networks to achieve superior results. In this project, one such pre-learning phase has been investigated and found to give significant improvement over the baseline performance obtained using raw pixel data as inputs to a classifier. The main aim of this project was to be able to find the relationship between the different hyper parameters of the feature learning model and the classification performance. This aim has been significantly achieved as is shown by the extensive study carried out in the project. 1 Problem Statement and Goals The project aimed to investigate the use of the unsupervised learning algorithm sparse auto encoders for the task of automatic feature learning on periodically divided image patches. The dataset chosen was a large image dataset CIFAR-10[1] comprising of 60,000 32×32 RGB images split into training, testing and validation sets. This is a multi class classification problem, where the number of classes is 10. The problem proved to be even more challenging to me because of the following modifications done to suit the goals of the course project: • The entire dataset was not chosen due to the size and time constraints of the project • An effort was made to study the effect of unsupervised learning of features in the reduced feature space, i.e., the grayscale space. • A large number of hyper parameters that needed to be tuned to achieve optimal performance. There were hyper parameters in the feature learning phase, and in the feature classification phase. Thus the problem seemed to have two black box testing phases. The end goal of this project was to learn how to tune the hyperparameters of a feature learning algorithm to achieve best performance in the task of classification. 2 Literature Review Several state-of-the-art image and object recognition researchers have obtained superior performance with their algorithms by including a pre-training unsupervised feature learning phase. The most important work in this field came from the authors in [2] where this feature learning was carried out on the MNIST handwritten digit recognition database. This method has also been used in activity recognition, speech recognition and other domains, where just a single layer of unsupervised learning improves the classification performance considerably [3, 5, 7]. The main work in this project is based on [4] where the feature learning is carried out using sparse-auto encoders. 3 Problem Formulation and Setup The feature learning is performed using an unsupervised learning algorithm called the sparse auto encoder. An auto encoder is a neural network based unsupervised learning algorithm that applied sparsity by means of backpropagation. The backpropagation is carried out by setting the values of the target equal to the input values. Thus, the netwrok tries to compress the input space in the hidden layer, and reconstructs the input at the output using the compressed