Paper:
Deep learning, a rebranding of deep neural network research works, has achieved a remarkable success in recent years. With multiple hidden layers, deep learning models aim at computing the hierarchical feature representations of the observational data. Meanwhile, due to its severe disadvantages in data consumption, computational resources, parameter tuning costs and the lack of result explainability, deep learning has also suffered from lots of criticism. In this paper, we will introduce a new representation learning model, namely "Sample-Ensemble Genetic Evolutionary Network" (SEGEN), which can serve as an alternative approach to deep learning models. Instead of building one single deep model, based on a set of sampled sub-instances, SEGEN adopts a genetic-evolutionary learning strategy to build a group of unit models generations by generations. The unit models incorporated in SEGEN can be either traditional machine learning models or the recent deep learning models with a much "narrower" and "shallower" architecture. The learning results of each instance at the final generation will be effectively combined from each unit model via diffusive propagation and ensemble learning strategies. From the computational perspective, SEGEN requires far less data, fewer computational resources and parameter tuning efforts, but has sound theoretic interpretability of the learning process and results. Extensive experiments have been done on several different real-world benchmark datasets, and the experimental results obtained by SEGEN have demonstrated its advantages over the state-of-the-art representation learning models.

Review:
This manuscript introduces SEGEN, a model based on Evolutionary Computation for building deep models. Interestingly, the authors define deep models in a different way. Instead of stacking several hidden layers one after the other (as in traditional deep learning models), SEGEN uses the idea of generations in evolutionary models (Genetic Algorithms or GA) and puts the unit models in the successive generations into layers, i.e., “evolutionary layer”. Each layer then performs the validation, selection, crossover, and mutation operations, as in GA. Another interesting point of the proposed method is that the choice of unit models in SEGEN can be traditional machine learning or recent deep learning models.
The paper touches an interesting topic and proposes a sound method. However, there are several issues with the paper. There are several ungrounded and untested claims, as well as many unclear points in the method.
-	In page 5, Section 4.2.4, the authors introduce the loss function used to define the fitness for the evolutionary model. It is not clear why they use the difference between the latent representations of the autoencoders (z) from pairwise nodes to define the loss. There are no motivations or discussion for this. Two different representations of two nodes may both be good (e.g., in terms of classification of data), but they do not have to be necessarily identical. 
-	Given the loss defined in Section 4.2.4, it is not clear how the authors ran their model for MNIST and other datasets, for which they used CNN and MLP unit models. In CNN and MLP there is not latent representation z.
-	Based on the model descriptions in Section 4.2 (and its Subsections), the proposed method transfers the learned models in previous generations to the next ones. But there is no explanation if the new models are again fine-tuned on the data? For instance, take the autoencoders, for two different unit models, the cross-over operator defuses the variables (weights and bias) from the two selected models to create an offspring. There is no guarantee that the new autoencoder model works properly on the same dataset. As a naïve example, if there are correlated and redundant features in the data, different autoencoders may separately focus on one/some of these features. Defusing weights of the two autoencoders (built upon different aspects of the data) may most probably ruin the whole model. 
-	There are four claims in the paper on the advantages of the proposed model, compared to other deep learning algorithms. None of these claims are discussed in depth or at least illustrated experimentally. 
*** Less Data for Unit Model Learning. The authors could have reported the number of variables used in each model in the experiments. It is important to see with how many of a larger number of variables a traditional deep model can result in comparable results to SEGEN. 
*** Less Computational Resources. The model operates in several generations and in each generation, many unit models are built. It is not fair to say and not clear how it can occupy less space or time complexity than a regular GCNN or MLP.
*** Less Parameter Tuning. Again experiments could clarify this issue.
*** Sound Theoretic Explanation. The authors only refer to (Rudolph 1994) for the performance bounds of their model and claim that since they are using GA they are better than other deep learning models. However, performance bounds for GA models are very shallow and proximal. 
-	To calculate the computational complexity of the model, the authors analyzed the time for learning one unit model. However, in GA models, the complexity is calculated using the bounds on the number of times the fitness function is called since the fitness function is the most computationally intensive task (please see: Pelikan and Lobo 1999 ‘Parameterless Genetic Algorithm A Worst-case Time and Space Complexity Analysis’). 
-	One of the main fallacies of GAs and evolutionary algorithms is that they may lead to premature convergence. This is very common, especially at the presence of trap functions, such as non-convex functions that real-world problems deal with (please see: Goldberg et al. 1991 ‘Massive Multimodality, Deception, and Genetic Algorithms’). There are no discussions/experiments on how SEGEN may overcome the premature convergence, or even if it converges at all.