Learning sparse feature representations is a useful instrument for solving an unsupervised learning problem. In this paper, we present three labeled handwritten digit datasets, collectively called n-MNIST by adding noise to the MNIST dataset, and three labeled datasets formed by adding noise to the offline Bangla numeral database. Then we propose a novel This is an extended version of the paper published in ESANN 2015, [1]. 123 856 S. Basu et al. framework for the classification of handwritten digits that learns sparse representations using probabilistic quadtrees and Deep Belief Nets. On the MNIST, n-MNIST and noisy Bangla datasets, our framework shows promising results and outperforms traditional Deep Belief Networks. Keywords Deep neural networks · Handwritten digit classification · Probabilistic quadtrees · Deep belief networks · Sparse feature representation Deep Learning has gained popularity over the last decade due to its ability to learn data representations in an unsupervised manner and generalize to unseen data samples using hierarchical representations. One of the popular unsupervised Deep learning models is the Deep Belief Network [2] . In [3], Deep Belief Networks have been used for modeling acoustic signals and have been shown to outperform traditional approaches using Gaussian Mixture Models for Automatic Speech Recognition (ASR). A greedy layerwise pre-training using Restricted Boltzmann Machine (RBM) is used to train Deep Belief Network. A sparse feature learning algorithm for Deep Belief Networks was proposed in [4] . However, their work was focused on maximization of information content in the learned representations. Restricted Boltzmann Machines, on the other hand, are trained by minimizing a contrastive term in the loss function. Deep Belief Networks have also been used for analyzing satellite imagery [5] as well as texture datasets [6] . The main contributions of our work are twofold -(1) We first present three labeled handwritten digit datasets, collectively called n-MNIST, and three labeled Bangla numeral datasets, created by adding white gaussian noise, motion blur and reduced contrast to the original MNIST dataset[7] and the offline handwritten Bangla numeral dataset [8] . (2) Then, we present a framework for the classification of handwritten digits that a) learns probabilistic quadtrees from the dataset, b) performs a Depth First Search on the quadtrees to create sparse representations in the form of linear vectors, and c) feeds the linear vectors into a Deep Belief Network for classification. On the MNIST, n-MNIST and noisy Bangla datasets, our framework shows promising results and outperforms traditional Deep Belief Networks. 1
For hand-written digit recognition, Basu et al. REF present the n-MNIST database, which is a modification of the benchmark MNIST dataset.
Learning Sparse Feature Representations Using Probabilistic Quadtrees and Deep Belief Nets
Neural Processing Letters
Neural Processing Letters
Computer Science