The deep convolutional neural network (CNN) is the state-of-the-art solution for large-scale visual recognition. Following some basic principles such as increasing network depth and constructing highway connections, researchers have manually designed a lot of fixed network architectures and verified their effectiveness. In this paper, we discuss the possibility of learning deep network structures automatically. Note that the number of possible network structures increases exponentially with the number of layers in the network, which motivates us to adopt the genetic algorithm to efficiently explore this large search space. The core idea is to propose an encoding method to represent each network structure in a fixed-length binary string. The genetic algorithm is initialized by generating a set of randomized individuals. In each generation, we define standard genetic operations, e.g., selection, mutation and crossover, to generate competitive individuals and eliminate weak ones. The competitiveness of each individual is defined as its recognition accuracy, which is obtained via a standalone training process on a reference dataset. We run the genetic process on CIFAR10, a smallscale dataset, demonstrating its ability to find high-quality structures which are little studied before. The learned powerful structures are also transferrable to the ILSVRC2012 dataset for large-scale visual recognition. It is worth emphasizing that the genetic algorithm is computationally expensive, as we need to undergo a complete network training process for each generated individual. We adopt the strategy to run the genetic process on a small dataset (CIFAR10), in which we observe the ability of the genetic algorithm to find effective network structures, and then transfer the learned top-ranked structures to perform large-scale visual recognition. The learned structures, most of which have been less studied before, often perform better than the manually designed ones in either small-scale or large-scale experiments. The remainder of this paper is organized as follows. Section 2 briefly introduces related work. Section 3 illustrates the way of using the genetic algorithm to design network structures. Experiments are shown in Section 4, and conclusions are drawn in Section 5.
Xie and Yuille REF have adopted genetic algorithms (GAs) to evolve CNN architectures by proposing a binary encoding method to represent GA individuals in a fixed-length string.
Genetic CNN
2017 IEEE International Conference on Computer Vision (ICCV)
2017 IEEE International Conference on Computer Vision (ICCV)
Computer Science