Distinction between features extracted using deep belief networks
Authors: mohammad pezeshki, Sajjad Gholami, Ahmad Nickabadi
ICLR 2014

Abstract: Data representation is an important pre-processing step in many machine learning algorithms. There are a number of methods used for this task such as Deep Belief Networks (DBNs) and Discrete Fourier Transforms (DFTs). Since some of the features extracted using automated feature extraction methods may not always be related to a specific machine learning task, in this paper we propose two methods in order to make a distinction between extracted features based on their relevancy to the task. We applied these two methods to a Deep Belief Network trained for a face recognition task.


Peer review:
[Review]:
This work proposed two methods to make a distinction between features learned by DBN. The authors seemed to imply that DBN could be capable of learning compositional features from the training data. However, I do not think DBN can learn those features easily. 
The method of variances is kind of counter-intuitive. While a average face feature could be activated by most of face images, the variance of its activation could be very small. According this method, the average face feature would be seen as a non-face node.