Learning Non-Linear Feature Maps, With An Application To Representation Learning
Authors: Dimitrios Athanasakis, John Shawe-Taylor, Delmiro Fernandez-Reyes
ICLR 2014

Abstract: Recent non-linear feature selection approaches employing greedy optimisation of Centred Kernel Target Alignment(KTA) exhibit strong results in terms of generalisation accuracy and sparsity. However, they are computationally prohibitive for large datasets. We propose randSel, a randomised feature selection algorithm, with attractive scaling properties. Our theoretical analysis of randSel provides strong probabilistic guarantees for correct identification of relevant features. RandSel's characteristics make it an ideal candidate for identifying informative learned representations. We've conducted experimentation to establish the performance of this approach, and present encouraging results, including a 3rd position result in the recent ICML black box learning challenge as well as competitive results for signal peptide prediction, an important problem in bioinformatics.


Peer review:
[Review]:
This interesting submission links prior work on KTA with the question
of representation learning.  I then provides learning theory to
underpin this idea and suggests a nonlinear feature selection
algorithm.  Finally, it shows its success for an ICML 2013 benchmark,
where the method finished with a bronze medal. In addition it shows
interesting/impressive results on bioinformatics, namely cleavage site
prediction. This is an interesting paper which should be accepted. 

Some suggestions/criticism that may help improving the ms:

1. it would be nice to briefly discuss runtimes

2. so far the LT results seem like a paper within a paper, it would be
nice to interweave this better, and maybe show a/the toy simulation
what the derived bounds mean on concrete data.

3. The algorithm seems slightly ad hoc, where do the 12.5% come from

4. It would be nice to leave more room to the applications, already
one seems sufficiently convincing

5. Finally, I strongly suggest to further tone down that statement
that the present approach is better than ref 14. I do not see this.
For this statement extensive testing with p-values and a thorough
understanding of the whys would be required.