Learning Non-Linear Feature Maps, With An Application To Representation Learning
Authors: Dimitrios Athanasakis, John Shawe-Taylor, Delmiro Fernandez-Reyes
ICLR 2014

Abstract: Recent non-linear feature selection approaches employing greedy optimisation of Centred Kernel Target Alignment(KTA) exhibit strong results in terms of generalisation accuracy and sparsity. However, they are computationally prohibitive for large datasets. We propose randSel, a randomised feature selection algorithm, with attractive scaling properties. Our theoretical analysis of randSel provides strong probabilistic guarantees for correct identification of relevant features. RandSel's characteristics make it an ideal candidate for identifying informative learned representations. We've conducted experimentation to establish the performance of this approach, and present encouraging results, including a 3rd position result in the recent ICML black box learning challenge as well as competitive results for signal peptide prediction, an important problem in bioinformatics.


Peer review:
[Review]:
The paper proposes a new and fast approach for selecting features using centered kernel target alignment.

As far as I can tell, there seems to be two contributions: 1. use kernel alignment as a way of selecting features 2. random subsample data so that the method can be made scalable. However, I think the algorithmic innovation seems to be thin.

In the description of the algorithm (section 4), the paper keeps referring to bootstrap (size). I think that is a misnomer --- the algorithm listing (Algorithm 1) changes to subsampling, which I think what the paper is using.  If the paper is indeed using boostrap (ie, sampling with replacement), the computational complexity would not be reduced as the # of samples (including repeated ones) will be unchanged.  

I do not follow the reasoning in Theorem 3.6 to come up with the 'bottom 12.5% ' features need to be thrown away.

The paper does not seem to explain what are the methods 'deep with XYZ' in Table 1. 

Overall, I think the writing of this paper could use some polishing.