Generative NeuroEvolution for Deep Learning
Authors: Phillip Verbancsics, Josh Harguess
ICLR 2014

Abstract: An important goal for the machine learning (ML) community is to create approaches that can learn solutions with human-level capability. One domain where humans have held a significant advantage is visual processing. A significant approach to addressing this gap has been machine learning approaches that are inspired from the natural systems, such as artificial neural networks (ANNs), evolutionary computation (EC), and generative and developmental systems (GDS). Research into deep learning has demonstrated that such architectures can achieve performance competitive with humans on some visual tasks; however, these systems have been primarily trained through supervised and unsupervised learning algorithms. Alternatively, research is showing that evolution may have a significant role in the development of visual systems. Thus this paper investigates the role neuro-evolution (NE) can take in deep learning. In particular, the Hypercube-based NeuroEvolution of Augmenting Topologies is a NE approach that can effectively learn large neural structures by training an indirect encoding that compresses the ANN weight pattern as a function of geometry. The results show that HyperNEAT struggles with performing image classification by itself, but can be effective in training a feature extractor that other ML approaches can learn from. Thus NeuroEvolution combined with other ML methods provides an intriguing area of research that can replicate the processes in nature.


Peer review:
[Review]:
Generative NeuroEvolution for Deep Learning
Phillip Verbancsics & Josh Harguess

Summary: Hyperneat neuroevolution is applied to MNIST, then fine-tuned through backprop. Results are perhaps not that overwhelming in terms of accuracy, but interesting.

Lots of relevant work seems to be missing though. For example, earlier indirect encodings were proposed by the following works:

A. Lindenmayer: Mathematical models for cellular interaction in development. In: J. Theoret. Biology. 18. 1968, 280-315.

H. Kitano. Designing neural networks using genetic algorithms with graph generation system. Complex Systems, 4:461-476, 1990.

C. Jacob , A. Lindenmayer , G. Rozenberg. Genetic L-System Programming, Parallel Problem Solving from Nature III, Lecture Notes in Computer Science, 1994

A universal approach to indirect encoding based on a universal programming language for encoding weight matrices with low Kolmogorov complexity:

J.  Schmidhuber. Discovering solutions with low Kolmogorov complexity and high generalization capability. In A. Prieditis and S. Russell, editors, Machine Learning: Proceedings of the Twelfth International Conference (ICML 1995), pages 488-496. Morgan Kaufmann Publishers, San Francisco, CA, 1995.

p 3:  'Such object recognition tasks where deep learning has achieved the best results include the MNIST hand-written digit dataset [30, 31], … and the ImageNet Large-Scale Visual Recognition Challenge [33]' 

But refs [30, 31] do not have best results on MNIST - as of 2012, the best result was in:

D. Ciresan, U. Meier, J. Schmidhuber. Multi-column Deep Neural Networks for Image Classification. Proc. IEEE Conf. on Computer Vision and Pattern Recognition CVPR 2012, p 3642-3649, 2012.

There may be even better results of 2013? And to my knowledge the best result on ImageNet is here:

M. D. Zeiler, R. Fergus. Visualizing and Understanding Convolutional Networks. TR arXiv:1311.2901 [cs.CV], 2013.

p 3: 'vanishing gradient' - why not cite the man who first discovered this:

S. Hochreiter. Untersuchungen zu dynamischen neuronalen Netzen. Diploma thesis, TUM, 1991

p 3: 'auto-encoders' - I think the authors mean stacks of auto-encoders - here one can cite:

[35] D. H. Ballard. Modular learning in neural networks. Proc. AAAI-87, Seattle, WA, p 279-284, 1987

p 4: 'In this way, HyperNEAT acts as a reinforcement learning approach that determines the best features to extract for another machine learning approach to maximize performance on the task'

So it is like Evolino, where evolution is used to determine the best features to extract for another machine learning approach to maximize performance - what is the main difference to the present approach? See:

J. Schmidhuber, D. Wierstra, M. Gagliolo, F. Gomez. Training Recurrent Networks by Evolino. Neural Computation, 19(3): 757-779, 2007.

p 7: 'NeuroEvolution approaches have been challenged in effectively training ANNs order of magnitude smaller than those found in nature'

Compare, however, vision-based RNN controllers with a million weights evolved through Compressed Network Search:

J. Koutnik, G. Cuccu, J. Schmidhuber, F. Gomez. Evolving Large-Scale Neural Networks for Vision-Based Reinforcement Learning. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO), Amsterdam, 2013. Also see lots of earlier work on this: http://www.idsia.ch/~juergen/compressednetworksearch.html

General recommendation: Interesting, but it should be made clear how this work is related to or goes beyond the previous work mentioned above.