Continuous Learning: Engineering Super Features With Feature Algebras
Authors: Michael Tetelman
ICLR 2014

Abstract: In this paper we consider a problem of searching a space of predictive models for a given training data set. We propose an iterative procedure for deriving a sequence of improving models and a corresponding sequence of sets of non-linear features on original input space. After finite number of iterations $N$ the non-linear features become $2^N$-degree polynomials on original space. We show that in a limit of infinite number of iterations derived non-linear features must form an algebra, so for any given input point a product of two features is a linear combination of features from same feature space. Due to convexity of each iteration and its ability to fall back to solutions found in previous iteration the models in the sequence have always increasing likelihood with each iteration while dimensionality of each model parameter space is set to a limited controlled value.


Peer review:
[Review]:
This paper presents an iterative way of defining increasingly complex feature spaces for prediction model selection. 
It is argued that in the limit, this sequence of features results in a feature algebra, meaning that products of features are linear combinations of features within the feature space. 

The mathematical ideas could be interested if explained in more clarity. 
The nonlinear super features look like an interesting way of reasoning of feature compositions, however, 

* I am not sure about the usefulness of the construction. 

* Version 1 of the paper is quite unpolished. 

MINOR:

In eq. (4) `maxarg' should be `argmax'. 
In eq. (9) use `langle' and `angle' instead of `<' and `>'. 
Eq. (6) seems not to be used in Section 3.