ENRICHMENT OF FEATURES FOR CLASSIFICATION USING AN OPTIMIZED LINEAR/NON-LINEAR COMBINATION OF INPUT FEATURES

Abstract: Automatic classification of objects is one of the most important tasks in engineering
      and data mining applications. Although using more complex and advanced
      classifiers can help to improve the accuracy of classification systems, it can be
      done by analyzing data sets and their features for a particular problem. Feature
      combination is the one which can improve the quality of the features. In this paper,
      a structure similar to Feed-Forward Neural Network (FFNN) is used to generate an
      optimized linear or non-linear combination of features for classification. Genetic
      Algorithm (GA) is applied to update weights and biases. Since nature of data sets
      and their features impact on the effectiveness of combination and classification
      system, linear and non-linear activation functions (or transfer function) are used
      to achieve more reliable system. Experiments of several UCI data sets and using
      minimum distance classifier as a simple classifier indicate that proposed linear and
      non-linear intelligent FFNN-based feature combination can present more reliable
      and promising results. By using such a feature combination method, there is no
      need to use more powerful and complex classifier anymore.

Reviews and author responses:

The paper presents a method for feature projection which uses a two level neural network like structure to generate new features from the input features. The weights of the NN like structure are optimised using a genetic search algorithm which optimises the cross-validation error of a nearest neighbor classifier. The method is tested on four simple UCI datasets. There is nothing interesting or novel about the paper. It is not clear whether the GA optimisation takes place on the level of cross validation error estimation or within an internal validation set as it should have been the case. The very high accuracies reported seem to hint the latter, which is a serious methodological error. The poor language and presentation does not help in clearing that, as it does not help in general. 

This paper proposes using a feedforward neural network (FFNN) to extract intermediate features which are input to a 1NN classifier. The parameters of the FFNN are updated via a genetic algorithm with a fitness function defined as the error on the downstream classification, on a held-out set. The performance of this approach is measured on several UCI datasets and compared with baselines.
– The paper’s main contribution seems to be a neural network with a GA optimization for classification that can learn “intelligent combinations of features”, which can be easily classified by a simple 1NN classifier. But isn't this exactly what neural networks do – learn intelligent combinations of features optimized (in this case, via GA) for a downstream task? This has already been successfully applied in multiple domains eg. in computer vision (Krizhevsky et al, NIPS 2011), NLP (Bahdanau et al 2014), image retrieval (Krizhevsky et al. ESANN 2011) etc, and also studied comprehensively in autoencoding literature. There also exists prior work on optimizing neural nets via GA (Leung, Frank Hung-Fat et al., IEEE Transactions on Neural networks 2003). However, this paper claims both as novelties while not offering any improvement / comparison. 
– The claim “there is no need to use more powerful and complex classifier anymore” is unsubstantiated, as the paper’s approach still entails using a complex classifier (a FFNN) to learn an optimal intermediate representation.
– The choice of activations is not motivated, and performance on variants is not reported. For instance, why is that particular sigmoid formulation used? 
– The use for a genetic algorithm for optimization is not motivated, and no comparison is made to the performance and efficiency of other approaches (like standard backpropagation). So it is unclear why GA makes for a better choice of optimization, if at all.
– The primary baselines compared to are unsupervised methods (PCA and LDA), and so demonstrating improvements over those with a supervised representation does not seem significant or surprising. It would be useful to compare with a simple neural network baseline trained for K-way classification with standard backpropagation (though the UCI datasets may potentially be too small to achieve good performance).
– The paper is poorly written, containing several typos and incomplete, unintelligible sentences, incorrect captions (eg. Table 4) etc.


The main issue is the scientific quality. What the authors call "intelligent mapping and combining system" for the proposed system is simply a fully connected neural network. Such systems have been largely investigated in the literature. The use of genetic algorithms has also been considered. Moreover, mapping features to some appropriate feature space has been widely investigated, including the choice of appropriate mapping. We didn't find anything "intelligent" in the proposed mapping. 

There are many spelling and grammatical errors.


Meta-review: The presented method essentially builds a model that remaps features into a new space that optimizes nearest-neighbor classification. The model is a neural network, and the optimization is carried out through a genetic algorithm.

Pros:
 - One major issue with neural network classification is that of a lack of explainability. Many networks are currently "black box" approaches. By moving to the optimization problem to that of building a feature space for nearest neighbor classification, one can, to a degree, alleviate the "black box" issue by providing the discovered nearest neighbor instances as "evidence" of the decision.
- Authors use established datasets.

Cons:
- Authors do not properly cite previous work, as brought up by reviewers. There is much literature on optimization of feature spaces (such as the entire field of metric learning), as well as prior approaches using genetic optimization. The originality and significance here is therefore not clear. 

Decision: rejected-papers