D EEP Neural Networks (DNNs), as the cornerstone of deep learning [1] , have demonstrated their great success in diverse real-world applications, such as image classification [2] , [3] , natural language processing [4] and speech recognition [5] . The promising performance of DNNs has been widely recognized as their deep nature, which is able to learn meaningful features directly from the raw data almost without any specialist intervention. Generally, the performance of DNNs depends on two aspects: their architectures and the weights, only when both achieve the optimal status simultaneously, the performance of the DNNs could be satisfied. The optimal weights are obtained through the learning process: using a continuous loss function to measure the difference between the real output and the desired output, and then the gradient-based algorithms are often used to minimize the loss. When the termination satisfied the condition which is commonly a maximal iteration number, the weights obtained at the termination are used as the optimum. Such kind of learning process has been very popular largely own to its effectiveness in practice, and has become the dominant technique for weight optimization [6] , although the gradient-based optimization algorithm is principally localsearch [7] . However, obtaining the architectures cannot be directly formulated by a continues function, and there is even no explicit function to measure the process for finding optimal architectures.
There has been a long time that the promising architectures of DNNs are manually designed with expertise. This can be seen from the state of the arts, such as VGG [8] , ResNet [2] and DenseNet [3] , to name a few. These promising CNN models are all manually designed by the researchers with rich knowledge in both neural networks and images processing. However, in practice, most end users are with limited expertise in DNNs. Moreover, the DNN architectures are often problem-dependent, if the distribution of the data is changed, the architectures must be redesigned accordingly. Neural Architecture Search (NAS), which aims to automate the architecture designs of deep neural networks, is recognized as an effective and efficient way to solve the limitations aforementioned.
Mathematically, the NAS is modeled by an optimization problem formulized by Equation (1):
where A denotes the search space of the neural architectures, L(·) measures the performance of the architecture A on the validation dataset D valid after being trained on the training dataset D train . The L(·) is always non-convex and nondifferential [9] . In principle, NAS is a complex optimization problem experiencing several challenges, e.g. complex constraints, discrete representations, bi-level structures, computationally expensive characteristics and multiple conflicting objectives. NAS algorithms refer to the optimization algorithms which are specifically designed to effectively and effectively solve the problem represented by Equation (1) . The source of "NAS" is generally recognized from the algorithm [10] proposed by Google, and its preprint version was first released in Arxiv in 2016 and then accepted for publication by the International Conference on Learning Representations in 2017.
Since then, a vast number of researchers have put concerns on proposing novel NAS algorithms. Existing NAS algorithms can be generally classified into three different categories based on the optimizer: Reinforcement Learning (RL) [11] based algorithms, gradient based algorithms and Evolutionary Computation (EC) [12] based NAS algorithms (ENAS). Specifically, the RL based algorithms often require thousands of Graphics Processing Cards (GPUs) performing several days even on the median-scale dataset, such as the CIFAR10 image classification benchmark dataset [13] . The gradient-based algorithms are more efficient than the RL based NAS algorithms. However, they often find the ill-conditioned architectures due to the relation is not mathematically proved. In addition, the gradient-based NAS algorithms require to construct a supernet in advance, which also highly requires expertise. The ENAS algorithms solve the NAS problems by using the EC technique. Specifically, the EC is a kind of population-based computational paradigm, simulating the evolution of species or the behaviour of the population in nature, to solve the challenging optimization problems, including the genetic algorithms [14] , genetic programming [15] , and particle swarm optimization [16] . EC has been widely applied to solve non-convex optimization problems [17] , and even the mathematical form of the objective function may not be available [18] .
In fact, more than twenty years ago, the EC methods have been frequently used for searching for not only the neural architecture but also the weights, which is also termed neuroevolution. The major differences between ENAS and the neuroevolution lie in two aspects. Firstly, the neuroevolution concerns on both the neural architectures and the weights while ENAS focuses on mainly the architectures. Secondly, the neuroevolution commonly takes effects on small-scale and median-scale neuron networks, while ENAS is mainly working on DNNs, such as the Convolutional Neural Networks (CNNs) [19] , [20] and deep stacked autoencoders [21] , which are the building blocks of deep learning [22] . The first work of ENAS is generally viewed as the LargeEvo algorithm [19] that is proposed by the Google Brain Team, who released its early version in March of 2017 in Arxiv, and then this paper got accepted by the 34th International Conference on Machine Learning in June of 2017. The LargeEvo algorithm employed the genetic algorithm to search for the best architecture of a CNN, and the experimental results on CIFAR-10 and CIFAR-100 [13] have demonstrated its effectiveness. Since then, a large number of ENAS algorithms have been proposed. Fig. 1 shows the number of submissions * focusing on the ENAS algorithms, from 2017 to the April of 2020 when we make this survey. It is clear to see the sharp increasing trend in the number of papers. Specifically, from 2017 to 2019, the number of submissions grows with multiple scales. During the first quarter of 2020, the submission is also the same number of 2018.
A great number of related submissions have been made available publicly, but there is no comprehensive survey of the literature on ENAS algorithms. Recent reviews on NAS can be seen in [18] , [23] , [24] and [25] . The first two reviews mainly focus on all kinds of methods of NAS, and make a macro summery on NAS. To be specific, Elsken et al. [23] divide NAS into three stages: Search Space, Search Strategy and Performance Estimation Strategy. Similarly, Wistuba et al. [24] also follow these three stages and add a review about the multiple objectives NAS. Darwish et al. [18] make a summary of Swarm Intelligence (SI) and EA approaches for deep learning. Nevertheless, this paper not only focuses on NAS, but also focuses on other hyperparameter optimization. * These submissions include the ones which have been accepted after peerreview and also the ones which are only available on the Arxiv website. $SULO Fig. 1 . The number of submissions focusing on evolutionary neural architecture search. The data is from Google Scholar with the keywords of "evolutionary" OR "genetic algorithm" OR "particle swarm optimization" OR "PSO" OR "genetic programming" AND "architecture search" OR "architecture design" and the literature on Neural Architecture Search collected by the AutoML.org website, until April 20, 2020. In addition, we have also carefully checked each collected manuscript to make its scope within the evolutionary neural architecture search.)
Stanley et al. [25] did a review of neuroevolution which is inspired by the development of the human brain and mainly aimed at the wight optimization rather than the structure. Most of the references in the surveys above are pre-2018 and do not involve a large number of papers in the past two years. This paper makes a summary involving a large number of ENAS papers to summarize current works and inspire some new ideas for promoting the performance of existing ENAS algorithms. The motivation of this paper can be divided into three aspects. First, a comprehensive survey is needed for a large number of ENAS literature. Several tables in this paper can make a general classification of the relevant papers, which makes the comparisons easier and more convenient. Second, the success of so many papers can attract more researchers from related research areas. Third, the advantage and disadvantage summarized from the state-of-the-art methods can give some enlightenment to the future development of ENAS algorithms. This paper also follows the three stages of NAS proposed by Elsken [23] , but some modifications are made to specifically suit ENAS algorithms which can be seen in Section II.
The reminder of this paper is organized as follows. The background of ENAS is provided in Section II. In Section III, the encoding strategy is introduced and different classification criteria of constraints on the encoding space are defined. Section IV summarizes the operators on the population to improve the search ability. Section V introduces the selection strategy and the way to speed up the evolution. Section VI presents the applications of ENAS algorithms. Section VII discusses the challenges and prospects, and Section VIII is for the conclusion.
ENAS is a process of neural architecture search using the EC methods. The performance of the architecture is getting better and better on the research task owing to the evolution of the population. Fig. 2 shows is a flowchart of ENAS process.
First of all, a population needs to be initialized. Each individual in the population represents a solution for the NAS, i.e., a neural architecture. Each architecture need to be encoded as an individual before it joins the population. There are two main questions in initialization stage: how to encode the architecture and how the original architecture generates. In the following, we answer these two questions by category. In history, there are two types of encoding method, namely the direct encoding and the indirect encoding. Direct encoding encodes all the information of the architecture exactly into an individual to undergo the evolving process. Direct encoding has two advantages: one is the convenience of encoding implement, the other is the flexibility of the operators on the population. On the contrary, the indirect encoding represents a generation rule for network architectures rather than representing the number and connections directly. [26] The indirect encoding is mainly used in the early ENAS method [27] and neuroevolution, e.g. HyperNEAT [28] . The main reason for choosing indirect encoding is to reduce the millions of neuron weight and connection, so that existing computing resources can meet the needs. Since back-propagation shows its superiority in optimizing the weight in neural network and many specific operators, such as convolution operator and pooling operator, have shown great ability dealing with computer vision task, very little information is required to build a neural network without considering any of the weight and connection. For example, only providing three hyper-parameters, i.e. kernel size, stride size and the number of filters can obtain a simple convolution layer. So, the few hyper-parameters make it reasonable to use direct encoding in ENAS in recent years. Commonly, there are fixed-length encoding strategy and variable-length encoding strategy based on whether the encoded string is of a fixed length. The advantage of the fixed-length encoding strategy is the ease of the population operations. In Genetic CNN [29] , for example, the fixed-length binary string helps the population operations (especially the crossover) become easier to implement. Another example is [30] , Loni et al. use a fixed-length string of genomes to represent the architecture. The advantage of this is that the one point crossover can be easily applied on the corresponding encoded information. The advantage of the variable-length encoding strategy is that it can contain more details of the architecture with more freedom of expression. However, the population operators may be not suitable for this kind of encoding strategy and need to be redesigned, where an example can be seen in [31] . In general, there are three types of original architecture generation: starting from trivial initial conditions [19] , randomly initialization in search space [31] and staring from a well-designed architecture (also termed as rich initialization) [32] . Generally, almost all the methods create only one population during the evolution. However, in [33] , the population is divided into three sub-populations which have its own population operators to hierarchically encode the architecture and be more efficiently searched. In addition, the population can be divided into several species (subgroups) which can maintain the diversity during the evolving process [34] , [35] .
Second, the generated individuals will undergo the fitness evaluation stage. Note that two fitness evaluation stages are in be defined first before the evaluation. Most ENAS methods define the fitness function as a criterion based on the task in hand, e.g., classification accuracy rate or error rate is suitable for image classification tasks and it can represent the performance of the corresponding architecture well. In general, many ENAS methods choose the direct way to obtain the fitness, i.e., after suffering the computationally expensive training phase, the true fitness will be calculated on the validation dataset which is not visible during the training phase. It will take hundreds of hours even on high-performance Graphics Processing Units (GPUs). Although many training tricks, such as weight inheritance [19] and early stopping policy [36] , are used to shorten training time, long evolving time due to the lack of a large amount of expensive computing resources remain a major obstacle to ENAS. The alternative way to obtain the fitness is to use a performance predictor which bypasses the process of the training phase during fitness evaluation. The existing performance predictors can be divided into two categories: predictors based on the learning curve and end-toend predictors [37] . The aim of the predictor is accelerating the fitness evaluation process under the condition of tolerable accuracy deviation. Thirdly, after the fitness evaluation on the initial population, it is an iteration based on the population. Until the stopping criterion is met, the population is updated by the population operators in each epoch of the iteration. Traditional EC approaches can guide the iteration, i.e. the evolving process. The operators on the current population are quite different in different EC approaches such as Genetic Algorithm (GAs), Particle Swarm Optimization (PSO) and Covariance Matrix Adaptation Evolutionary Strategies (CMA-ES), but all the operators can be classified into two categories: single individual based and multiple individuals based. The single individual based operators aimed at exploring the adjacent area of the individual. The mutation operator in GAs is a typical singe individual based operator. The multiple individual based operators take the use of two or more individual's information from the population, and the crossover operator in GAs is a representative. The population operators play an important role in ENAS, because it directly determines what new individuals are generated and what architecture can be explored in the process of evolution.
Finally, with the end of the iteration, a population suffering the evolution is obtained. For most ENAS method, the aim is to find a way simulating the process of expert designing neural network. So they just select the best individual from the last population as the output. Instead of discarding individuals in the last population, Frachon et al. [36] propose a Neural Committee to contain all the individuals in the last population, and make a decision by the voting from the Neural Committee. Furthermore, multi-objective ENAS methods return a set of architectures in the Pirato Front.
This section will discuss the first stage of the ENAS, namely the population initialization. This is the first stage in the evolving process, and is before the iteration. So all the global settings including hyper-parameters of evolution (the number of evolving epoch, the size of population, etc.), the type of individual presentation, the encoding space, etc., must be decided in this first stage. Since mentioned in Section II, most of the recent methods choose direct encoding strategy, we do not discuss the encoding strategy in this section any more. In general, two aspects: individual presentation and encoding space, can distinguish between different algorithms. The two aspects are introduced in the remainder of this section.
An individual in the population presents a corresponding a neural network architecture, and it must contain all the information about the architecture. As neural network is made up by basic units which are demonstrated in Section III-C, such as layers, blocks and cells, two aspects information can be used to build up a structure, that is, the parameters in each basic unit and the connections. The parameters are always the real numbers which can be directly encoded into an individual. Therefore, we only discuss the connections in this section.
One special case is the network connection is totally linear, this means there is no need to solely encode the information about the connections, and only the parameters in each basic unit are enough. One classical example can be seen in [31] where Sun et al. explore a great many of parameters based on a linear CNN mode. However, most architecture is not designed to be linear. The skip connection in ResNet [2] and the dense connection in DenseNet [3] show the ability to build a good architecture. The following lists two types of encoding connection.
1) Adjacent matrix: Because each unit can be regarded as a vertice and each connection can be regarded as an edge, the adjacent matrix can be easily applied to represent the connection. Genetic CNN [29] uses a binary string to represent the connection, and the string can transform into a triangular matrix. In the binary string, 1 denotes there is a connection between the two nodes and 0 denotes no connection. Lorenzo et al. [38] use a matrix to represent the skip connection, and this work revolves around the adjacent matrix. In fact, back in the 1990s, Kitano et al. [27] began to study the use of the adjacent matrix to represent network connection and explained the process from the connectivity matrix, to bit-string genotype, to network architecture phenotype.
2) Directed acyclic graph: Another way to present the connection is based on the directed acyclic graph. Irwin et al. [39] use a graph-based strategy to present the connection. To be specific, an ordered pair G = (V, E) with vertices V and edge E associated with a direction can describe a directed acyclic graph. Byla et al. [40] using a changing graph to represent the connection, and at the same time, they demonstrate Ant Colony Optimization (ACO) is suitable for the graph based presentation. Chiu et al. [41] proposed the directed acyclic graph neural networks, and showed the relation between adjacent matrix and the directed acyclic graph.
Encoding space contains all the possible individuals. Therefore, the output of ENAS is also an individual selected in the encoding space. Generally, the encoding space can be divided into two parts, the first one is the initial space, and the second is the search space. The initial space determines which kind of individuals may appear in the initial population, and the search space determines what kind of individual will appear in the iteration, or the evolving process. 1) Initial space: As mentioned in Section II, there are three different types of original architecture generation, i.e., three types of initial space: the trivial space, the random space and the well-designed space. The trivial space contains very simple architectures. For example, Large-Scale Evolution [19] initializes the population in a trivial space where each individual constitutes just a single-layer model with no convolutions. Xie et al. [29] also did the experiment to prove that a trivial space can evolve to a competitive architecture. The reason using as little experience as possible is to force the evolution to make discoveries by itself and to prove the evolution is not manipulated by the experimenter. On the contrary, the welldesigned space contains the state-of-the-art architectures. In this way, a promising architecture can be obtained, whereas it can not explore other novel architectures. For random space, all the individuals in the initial population are randomly generated in the limited space, and many methods adopt random space, such as [9] , [31] , [42] . The aim of this type of initial space is also to reduce the intervention of artificial experience in the initial population.
2) Search space: In the evolving process, the space for constraining individuals will change into the second space, the search space. Generally speaking, the methods choosing random initial space as the initial space have the same space in evolving process, i.e., the initial space and the search space is the same in these methods. For other two types of initial space, however, due to the relatively small initial space, the search space will become larger to contain the promising architectures. It's worth noting that many methods do not directly define the search space, but give the operators on population.
The constraints on the encoding space is important, because the constraints represent the human intervention which restricts the encoding space and is inclined to design an architecture similar to the known state-of-the-art one. A method with a mass of constraints can obtain an awesome architecture easily but prevent to design a novel architecture. Furthermore, the different size of search space would greatly affect the efficiency of evolution. We can not judge an ENAS method without its constraints on the search space, because one extreme case is that all the individuals in the search space are well performed. In this case, an excellent individual can be obtained even if do no selection. So we will discuss the constraints in detail in this section. Table I shows the different categories in encoding space and the constraints where horizontal classification is based on the constraints on the architecture, and vertical classification is based on the categories of the basic units searched. We will introduce the encoding space from this two kinds of classification.
First, the basic units can be divided into four categories: global, block-based, cell-based and topology-based. The global denotes the basic units in the encoding space are layers (such as convolution layers and pooling layers) or neuron nodes. The individual in a global encoding space is much more delicate, because every detail can be considered. However, it may take more time to obtain a delicate individual in such a large global space. So, many researchers use block which is a specific structure combination of various types of layers as the basic unit. Many different kinds of blocks are presented: ResBlock [2] , DenseBlock [3] , ConvBlock (Conv2d + BatchNormalization + Activation) [111] and InceptionBlock [139] etc. These blocks have promising performance and fewer parameters are needed to build the architecture. So it is easier to find a good architecture in the block-based encoding space. Some methods use these above blocks directly, such as [9] , [36] , and some other methods propose other blocks for their own purposes. Chen [107] et al. propose 8 blocks including ResBlock and InceptionBlock encoded in 3 bits string, and use Hamming distance to tell the similar blocks. Song [113] et al. propose three residual dense blocks to reduce the amount of computation due to the convolution operation of image super-resolution tasks. Cell-based encoding space is similar to the block-based space, and it can be regarded as a special case in blockbased space where all the blocks are the same. The methods choosing this space build the architecture by stacking repeated motifs. Chu et al. [127] divides the cell-based space into two irrelevant parts: the micro part contains the parameters of cells while the macro part defines the connections between different cells. The cell-based space greatly reduces the search space, but Frachon et al. [36] believes that there is no theoretical basis for that the cell-based space can help to get a good architecture. In the last, the topology-based space does not care about the parameters or the structure of each unit (layer or block), and their only concern is the connections between units. One classical example is the One-Shot which treats all the architectures as different subgraphs of a supergraph [23] . Yang et al. [135] search the architecture by choosing different connections in the supergraph, the subgraph built by the chosen connections becomes an individual. Another typical case is pruning. Wu et al. [132] shallow VGGNet [8] on CIFAR-10, and the aim is to prune unimportant weight connections. To sum up, the basic unit can be regarded as a special constraint on the encoding space. The block-based space makes the method more efficiency whereas the global space provides greater possibility to reach a novel architecture.
Secondly, the constraints on the encoding space mainly focus on three aspects: fixed depth, rich initialization and partial structure fixed. The fixed depth means all the individuals in the population have the same depth. The fixed depth is a strong constraint and largely reduce the encoding space. Noting that the fixed-length encoding strategy mentioned in Section II is different from the fixed depth. In Genetic CNN [29] , for example, the fixed-length encoding strategy only limits the maximum depth. The node which is isolated (no connection) is simply ignored. By this way, the individuals can obtain different depth. The second is rich initialization, and this is also a strong constraint with a lot of manual experience. In this case, the generated architecture is around the original one, and the possibility of exploring new architectures is greatly reduced. The partial structure fixed means the architecture is partially settled. For example, in [60] a max-pooling layer is added to the network after every set of four convolution layers.
In Table I , the relatively few constraints denotes the method has no restrictions on these above three aspects, but that is not to say there is absolutely no constraint. For example, in the classification task, fully connected layer is used as the last layer in some methods [31] , [79] , [125] , or the softmax crossentropy loss is used as the loss function [26] . Furthermore, there are other constraints. Supernet is a constraint, Yang et al. [135] devote to find a Subnet in the Supernet. As a result, the architecture found cannot be the one that is not presented in the Supernet. The best individuals found by some methods cannot be applied directly to the corresponding task until they undergo the subsequent operation such as training from scratch [105] , [120] or fine tuning [40] . Moreover, the maximum length is predefined in many methods including both fixed-length encoding strategy [29] and variable-length encoding strategy [31] resulted in preventing the method discovering a deeper architecture. Wang et al. [71] try to break the limit of maximum length by using a Gaussian distribution initialization mechanism. Irwin et al. [39] break the limit not by initialization, but using the population operators, the crossover 
Rich initialization Partial structure fixed Relatively few constraints Global [32] , [43] - [53] [32], [54] - [58] [42], [50] , [59] - [69] [19], [21] , [31] , [36] , [40] , [70] - [105] Block-based [106] - [108] [109] [65] , [109] , [110] [9], [20] , [26] , [30] , [36] , [105] , [111] - [120] Cell-based [121] - [123] [120], [124] - [131] Topology-based [132] , [133] [29], [38] , [134] - [138] and the mutation, to extend the depth to any size.
In this section, we will introduce the population operators. The population operator is performed on the population to generate offspring. The population operator can be regarded as the search strategy in the ENAS. The population is evolving to a better one with the guidance of the population operators during the evolving process. Generally, the operators can be divided into two categories: the single individual based and the multiple individuals based. One classical single individual based operator is the mutation operator in GA. It allows an individual search the architecture in the space around itself. The multiple individuals based operators take the use of the encoded information from more than one individual. The crossover operator in GA combines the encoded information from both parents. Furthermore, the location update mechanism in PSO takes the use of both the particle † own best configuration achieved in the past (P best ), and which particle is the current global best in the swarm (G best ). We want to discuss the population operators from these two aspects in this section.
First, we want to discuss the mutation operator. The aim of the mutation operator is to search the local optimum around the individual. A simple idea is to allow the encoded information to vary from a given range. Sun et al. [31] use the polynomial mutation [140] on the encoded information which is expressed by real numbers. To make the mutation not random, Lorenzo et al. [38] proposed a novel Gaussian mutation based on a Gaussian regression to guide the mutation, i.e., the Gaussian regression can predict which architecture may be good, and the new generated individuals are sampled in the regions of the search space where the fitness values are likely to be high. That makes the mutation have a "direction". Maziarz et al. [141] use a recurrent neural network (RNN) to guide the mutation. In this work, the mutations are not sampled at random among the possible architectural choices, but are sampled from distributions inferred by an RNN. Actually, using an RNN to control the mutation can be seen in other methods such as [127] . Qiang et al. [72] use a variable mutation probability. They used a higher probability in the early stage for better exploration and a lower probability in the late stage for better exploitation. The fact that it is applied in many other methods [66] is enough to prove its effectiveness. To maintain the diversity of † The individual in PSO is called particle. the population after the mutation, Tian et al. [42] use force mutation and distance calculation which ensure the individual in the population is not particular similar to other individual (especially the best one). Kramer et al. [126] use the (1+1)-ES that generates an offspring based on a single parent with bit flip mutation, and use mutation rate control and niching to overcome local optima. Zhang et al [104] propose the exchange mutation which exchanges the position of two genes of the individual, i.e. exchange the order of layers. This will not bring new layers and the weight can completely be preserved.
Chen et al. [142] introduced two function-preserving operators on the neural networks, and the function-preserving operators on the neural networks are termed as network morphisms by Wei et al. [143] . The network morphisms (function-preserving operators) aim to change the neural network architecture without loss of the acquired experience. They function-preserving operator change the architecture from F (·) to G(·), and satisfies the Equation 2:
where x denotes the input of network. The network morphisms can be regarded as a function-preserving mutation. With this function-preserving mutation, the mutated individual can not have a worse performance than the parent. Using the network morphisms in ENAS is suitable, because both are the incremental learning. To be more specific, Chen et al. [142] proposed net2widernet to obtain a wider net and net2deepernet to obtain a deeper net. Elsken et al. [111] extent the network morphisms with two popular network operations: skip connections and batch normalization. Zhu et al. [56] proposed five well-designed function-preserving mutations to guide the evolving process by the information have already learned. To avoid the suboptimal, Chen et al. [121] add noises in some function-preserving mutation, and in the experiment they found that by adding noises to pure network morphism, instead of compromising the efficiency, it will, by contrast, improve the final classification accuracy. Noting that all the network morphisms can only increase the capacity of a network, because if one would decrease the networks capacity, the function-preserving property could in general not be guaranteed [105] . And as a result, the architecture generated by network morphisms is only going to get larger and deeper which is not suitable for a device with limited computing resources (like a mobile phone). In order for the network architecture to be reduced, Elsken et al. [105] proposed the approximate network morphism, which satisfies the Equation 3 ,
to also cover operators that reduce the capacity of a neural architecture.
The multiple individual based operator can be divided into two categories according to whether it is directed or not: directed multiple individual based operator and the undirected multiple individual based operator. The location update mechanism in PSO is a classical example of the directed multiple individual based operator. The particle is moving toward the G best and the P best . On the contrary, there is no clear direction between individuals in the undirected multiple individual based operator. For example, the crossover operator in GA is an undirected multiple individual based operator, because the individuals in this operator have the equal status.
For the crossover operator, Sun et al. [31] using the Simulated Binary Crossover (SBX) [144] to do a combination of the encoded parameters from two matched layers. Instead of the crossover only focus on the parameters of layers, Sapra et al. [76] proposed a disruptive crossover swapping the whole cluster (a sequence of layers) between both the individual at corresponding positions.
For PSO, the individual in the population is called the particle. The position X of each particle represents the encoded information. The particle velocity V is updated according to the Equation 4:
where V t+1 i,j denotes the updated j-th dimension of the i-th particle's velocity, w is inertial coefficient of speed, c 1 and c 2 are constant that are used to fine-tune the performance of PSO, r 1 and r 2 are random numbers in [0,1), X is the current particle position. The position is updated via the Equation 5:
Junior et al. [70] use their implementation of PSO to update the particle based on the layer instead of the parameters of the layer. Gao et al. [145] developed a gradient-priority particle swarm optimization to handle the problem including the low convergence efficiency of PSO when there are a lot of hyperparameters to be optimized. They expect the particle tends to find the locally optimal solution at first, and then move to the global optimal solution. The Firefly algorithm is similar to PSO. Sharaf et al. [74] use an improved Firefly algorithm where the individuals in the population not only move toward the other individual better than itself, but also have the tendency for random mutations which give it the ability to search for local optimum.
For ACO, the individuals are generated in a quite differently way. Several ants are in an ant colony ‡ . Each ant move ‡ The population in ACO also termed as colony. from node to node following the pheromone instructions to build an architecture. The pheromone is updated generation after generation. The paths of well-performed architecture will maintain more pheromone to attract the next ant for exploitation and at the same time the pheromone is also decaying (pheromone evaporation) which encourage other ants to explore other areas. Byla et al. [40] let the ants choose the path from node to node in a graph whose depth increases gradually. Elsaid et al. [146] introduce different ant agent types to act according to specific roles to serve the needs of the colony (population), which is inspired by the real ants specialize.
The Covariance Matrix Adaptation-Evolution Strategy (CMA-ES) is widely used [54] , [147] . CMA-ES worked by adapting a covariance matrix C which defines the shape and orientation of a Gaussian distribution and vector x that describes the location of the center of the distribution [147] . The sampled individuals are used to update the C and the x which takes the use of multiple individuals information.
The Differential Evolution (DE) is another method whose population operators are all the multiple individual based. Different from the GA, the mutation in DE takes the use of the information from three individuals, and the mutated individual in terms of vector is generated via the Equation 6 :
where X G 1 , X G 2 , X G 3 are different individuals selected randomly from the population, F is an amplification factor. And then, the crossover operator in DE generates the experimental individual via Equation 7:
where j = {1, 2, ..., D}, D denotes the length of vector (chromosome), C r is predefined crossover rate constant, j rand is randomly chosen from [1, 2, ..., D] to ensure that at least one component of the experimental individual comes from the mutated individual. Some ENAS method like [48] , [87] choose DE to guide the offspring generation.
Moreover, Wang et al. [148] proposed a hybrid PSO-GA method. They use PSO to guide the evolution of the parameters in each block encoded in decimal notation, meanwhile use GA to guide the evolution of the shortcut connections encoded in binary notation. Table II is a classification of the different ENAS methods according to the EC method they based, and the horizontal classification are different types of neural networks.
There are some other methods not mentioned above. Hill climbing algorithm can be interpreted as a very simple evolutionary algorithm. For example, in [111] the population operators only contains the mutation and no crossover, and the selection strategy is relatively simple. The memetic algorithm is the hybrids of EAs and local search. Evans et al. [149] integrate the local search (as gradient descent) into the GP as a fine-tuning operation. The CVOA [85] is inspired by the new respiratory virus, COVID-19. The architecture is found by simulating the virus spreads and infects healthy individuals.
Hyper-heuristic contains two levels: high level strategy and low level heuristics, and a domain barrier is between these two levels. So the high level strategy is still useful when the applications have changed. AIS is inspired by theories related to the mammal immune system and do not require the crossover operator compared to the GA [36] .
Noting that, this section only discusses the single object optimization and focuses on how the population operators work to generate offspring. The multiple objective is discussed in Section V.
In this section, the evaluation for individual and the selection strategy is briefly summarized first. Due to the evaluation is the most time-consuming stage, we will discuss the strategies to accelerate this stage.
In EA, the population goes through this iteration: after offspring generation, the better individuals survive and the others are discarded. While in SI, the best individual must be found out. Nevertheless, each method needs a criterion to evaluate an individual. Different methods choose the criteria according to the task at hand. For example, the classification task prefers the classification accuracy as the criterion. In EA, this criterion is termed as fitness, while in AIS, this criterion is termed as affinity § .
In Table II , methods are divided into single object and multiple object first. The single object means there is only one objective function in the method, e.g., the classification accuracy mentioned above, and these methods have only one object: searching the architecture with the highest accuracy. The multiple object, however, contains more than one objective functions, e.g., aiming at both classification accuracy and computational costs simultaneously. However, these objective functions are always conflicting, i.e. getting a higher accuracy often requires a more complicated architecture with the need of more computational resources, on the contrary, on a device with limited computational resource, e.g. a mobile phone, can not afford such complex architecture. To this end, computational expense is one of the most important factors to consider. Pruning is a special case of computational expense working by cutting out the unimportant connections with minimal impact on accuracy. Other ENAS methods aiming at multiple object optimization can be seen in Table II . For example, Schorn et al. [78] also take the high error resilience into consideration.
The simplest way to tackle the multi objective optimization is by converted it to the single objective optimization. The Equation 8
is the classical linear form combining two objective function f 1 , f 2 into a single objective function where the λ denotes the coefficient. In [33] , [61] , [80] , [112] , the multi-objective optimization problem can be easily solved by using the available § In this paper, we call the evaluation criterion as "fitness". single objective optimization methods by the Equation 8 of the weighted sum. Chen et al. [128] do not adopt the linear addition as the objective function, whereas using a nonlinear penalty term. Some algorithms have been already widely used in multiobjective optimization, such as NSGA-II [178] , MOEA/D [179] , which are also used in [30] , [106] , [172] . These methods aim to find a Pareto-font set (or non-dominant set). We only put these methods in the multiple objective category of the Table II . Baldeon et al. [106] choose the penalty based boundary intersection approach in MOEA/D because training a neural network involves non-convex optimization and the form of Pareto Front is unknown. LEMONADE [105] divide the objective function into two categories: f exp and f cheap . The f exp denotes expensive-to-evaluate objectives (e.g., the accuracy), the f cheap denotes cheap-to-evaluate objectives (e.g., the model size). In every iteration, they sample parent networks with respect to sparsely distribution based on the cheap objectives f cheap to generate offspring. Therefore, they evaluate the f cheap more times than f exp to save time. Schoron et al. [78] also take the use of the LEMONADE proposed by Elsken et al. [105] . Due to the NSGA-III [180] may fall into the small model trap (this algorithm prefer the small models), Yang et al. [135] have made some improvements to the conventional NSGA-III for protecting these larger models. Li et al. [47] use the biasvariance framework on they proposed multi-objective PSO to get a more accurate and stable architecture. Wu et al. [132] use the MOPSO [181] for neural networks pruning. Wang et al. [108] use the OMOPSO [182] which selects the leaders using a crowding factor and the G best is selected from the leaders. Table III shows the main kinds of selection strategy. Note that the selection strategy often appears in EA based method, because other algorithms such as PSO and ACO do not need to undergo the selection stage, or the selection strategy can be regarded as integrated into the population operators. And the selection strategy can be not only used in environment selection stage which is choosing individuals to make up next population, but also used in choosing individuals as parents to generate offspring with the population operators. Zhang et al. [104] term these two selections as environmental selection and mate selection separately.
The selection strategy can be divided into five categories: retain the best group, discard the worst, roulette, tournament selection and others. The simplest strategy is to retain the individuals with higher fitness. Only the best group can survive. However, this can cause a loss of diversity in the population which may lead the population fall into local optima. Discarding the worst is similar to retaining the best group. Real et al. [130] using the aging evolution which discards the oldest individual in the population. Aging evolution can explore the search space more, instead of zooming in on good models too early, as non-aging evolution would. The same selection strategy is used in [100] . Zhu et al. [56] combine these two approaches, discarding the worst individual and the oldest individual at [35] , [41] , [81] , [138] , [150] [9], [19] , [20] , [29] , [31] , [32] , [42] , [44] , [46] , [49] , [53] , [56] - [58] , [60] , [62] - [64] , [66] - [68] , [71] , [76] , [79] , [80] , [83] , [84] , [86] , [88] , [90] - [92] , [94] , [96] , [98] , [99] , [104] , [107] , [110] , [113] , [116] - [118] , [120] , [121] , [124] - [126] , [130] , [131] , [137] , [151] , [152] [100], [153] [65], [91] , [93] , [154] , [155] [156]
GP [157] [26], [59] , [112] , [114] , [158] , [159] [34], [160] [161]
ES [138] , [162] [122] [54] , [73] , [147] , [154] , [163] , [164] [97] SI ACO [40] [134], [136] , [146] , [165] PSO [52] , [82] , [166] [70], [71] , [95] , [102] , [103] , [108] , [109] , [129] , [145] [72], [167] [21], [43] Firefly algorithm [74] Other Hill climbing algorithm [55] , [77] , [111] Memetic [38] , [ [36] Multiple object Computational expense [169] - [171] [30], [61] , [69] , [89] , [101] , [105] , [106] , [119] , [127] , [135] , [172] - [175] [176]
Pruning [30] , [132] , [133] Other [47] , [78] , [177] [ 51] the same time. Roulette gives every individual a probability to survive (or be discarded), whether he is the best or not. Tournament selection selects the best one from an equally likely sample of individuals. Furthermore, Johner et al. [57] use a ranking function to choose individuals by rank. A selection trick termed as niching is used in [67] , [126] to avoid stacking into local optima. This trick allows offspring worse than parent for several generations until evolving to a better one.
Most of the methods focus on preserving the well-performed individuals, however, Liu et al. [124] emphasizes the gene more than the survived individuals where gene can represent any ingredient in the architecture. They believe the individuals consist of the fine-gene set are more likely to have a promising performance.
In fact, some selection methods aim at preserving the diversity of the population. Elsken et al. [105] selects individuals in inverse proportion to their density. Javaheripi et al. [183] choose the parents based on the distance (difference) during the mate selection. They choose the two individuals have the highest distance to promote exploration.
Real et al. [19] use 250 workers (computers) to finish the Large-Scale evolution over 11 days. Such computational resources are not available for anyone who is interested in NAS. Retain the best group [26] , [36] , [38] , [42] , [49] , [55] , [73] , [77] , [84] , [97] , [111] , [114] , [132] , [156] , [157] Discard the worst or the oldest [56] , [76] , [86] , [130] , [131] , [155] , [184] Roulette [29] , [30] , [44] , [60] , [62] , [107] , [113] , [116] , [133] Tournament selection [9] , [19] , [20] , [31] , [59] , [63] , [80] , [93] , [99] , [108] , [117] , [120] , [125] , [127] , [128] , [184] Others [57] , [105] Almost all of the methods evaluate individuals by training them first and evaluating them on the validation dataset (i.e. a dataset which is unseen during the training). Since the architecture is becoming more and more complex, it will take a lot of time training each architecture to convergence. So it is natural to think up the methods to shorten the evaluation time. Table IV lists four of the most common methods to shorten the time: weight inheritance, early stopping policy, reduced training set and reduced population. Because the population operators usually do not completely disrupt the architecture of an individual, some parts of the new generated individual are the same with previous individuals.
The weights of the same parts can be easily inherited. With the weight inheritance, the neural networks are no longer trained completely from scratch. This method has been used in [166] 20 years ago. Moreover, as mentioned in Section IV, the network morphisms change the network architecture without loss of the acquired experience. This could be regarded as the ultimate weight inheritance, because it solved the weight inheritance problem in the changed architecture part. The ultimate weight inheritance let the new individual completely inherit the knowledge its parent learned and training such an individual to convergence will save a lot of time.
Early stopping policy is another method which is used widely in NAS. The simplest way is to set a fixed relatively small number of training epochs. This method is used in [21] , because the authors think the individual after a small number of training epochs can conduct the performance. Similarly, Assunccao et al. [79] let the individuals undergo the training for a same and short time each epoch (although this time is not fixed and will increase with the epoch). To let the promising architecture have more training time to get a more precise evaluation, So et al. [185] set hurdles after several fixed epochs. The weak individuals stop training early and save the time. However, the early stopping policy can lead to inaccurate estimation about individual (especially the large and complicated architecture) performance, which can be seen in Fig. 3. In Fig. 3 , individual2 performs better than individual1 before epoch t1, whereas individual1 performs better in the end. Yang et al. also discuss this phenomenon in [135] . So, it is crucial to determine at which point to stop. Note that the neural network can converge or hardly improve its performance after several epochs, such as the t1 for individual2 and the t2 for individual1 in Fig. 3 . Using the performance estimated at this point can evaluate an individual relatively accurately with less training time. Therefor, some methods such as [32] , [86] stop training when observing there is no significant performance improvement. Suganuma et al. [114] use the early stopping policy based on a reference curve. If the accuracy curve of an individual is under the reference curve for successive epochs, then the training will be terminated and this individual is regarded as poor one. After every epoch, the reference curve is updated by the accuracy curve of the best offspring.
Reduced training set, i.e. using a subset of that data has similar properties to a large dataset can also shorten the time effectively. Liu et al. [124] explore the promising architecture by training on a subset and use the transfer learning to the large original dataset. Because there are so many benchmark datasets in image classification field, the architecture can be evaluated on the smaller dataset (e.g. CIFAR-10) first and then is applied on the large dataset (such as CIFAR-100 and ImageNet). The smaller dataset can be regarded as the proxy for the large one.
Reduced population is the special method in ENAS, since other NAS do not have the population. Assunccao et al. [98] reduce the population on the basis of their previous algorithm [186] , [187] to speed up the evolution. However, simply reduced population may not explore the search space dramatically in each epoch and may lose the global search ability. Another way is reducing the population dynamically. For instance, Fan et al. [122] use the (µ + λ) evolution [19] , [36] , [38] , [40] , [55] , [56] , [62] , [67] , [77] , [78] , [84] , [86] , [88] , [105] , [109] , [111] , [120] , [121] , [125] , [132] Early stopping policy [21] , [32] , [36] , [42] , [43] , [62] , [65] , [71] , [79] , [80] , [83] , [86] , [86] , [95] , [98] , [103] , [108] , [112] , [114] , [125] , [145] Reduced training set [46] , [84] , [124] , [129] , [188] Reduced population [98] , [122] , [124] ACC epoch t1 t2
Individual 1 Individual 2
strategy and divide the evolution into three stages with the population reduction which aim to find the balance of the limited computing resources and the efficiency of evolution. The large population in the first stage is to ensure the global search ability, while the small population in the last stage is to shorten the evolution time. Instead of reducing the population, Liu et al. [188] evaluate the architecture with small size at an early stage of evolution. Similarly, Wang et al. [129] do not evaluate the whole architecture whereas a single block, and then the blocks are stacked to build an architecture as evolution goes on. Actually, there are many other well performed methods to reduce the time in ENAS. In population based methods, especially the GA based methods (e.g., in [31] ), it is natural to maintain well performed individuals in the population in successive epochs. Sometimes, the individuals in the next population directly inherit all the architecture information of their parents without any modification and it is not necessary evaluating the individuals again. Fujino et al. [32] use a memory to record the fitness of individuals, and if the same architecture encoded in individual appears, the fitness value is retrieved from memory instead of being reevaluated. Similarly, Miahi et al. [117] and Sun et al. [20] , employ a hashing method for saving pairs of architecture and fitness of each individual and reusing them when the same architecture appears again. Johner et al. [57] prohibit the appearance of architectures that have existed before in offspring. This does reduce the time, however, the best individuals are prohibited from remaining in the population which may lead the population evolving towards a bad direction.
Rather than training thousands of different architectures, oneshot model [189] trains only one SuperNet to save the time. The different architectures, i.e. the SubNets, are sampled from the SuperNet with the shared parameters. Yang et al. [135] believe the traditional ENAS methods without using SuperNet are less efficient for models are optimized separately. In contrast, the one-shot model optimize the architecture and the weights alternatively. But the weight sharing mechanism brings a difficulty in accurately evaluating the architecture. Chu et al. [190] scrutinize the weight-sharing NAS with a fairness perspective and demonstrate the effectiveness. However, there remains some doubts that can not explain clearly in oneshot model. The weights in the SuperNet are coupled. It is unclear why inherited weights for a specific architecture are still effective [137] .
Taking the use of hardware can reduce the time, too. Jiang et al. [89] use a distributed asynchronous system which contains a major computing node with 20 individual workers. Each worker is responsible for training a single block and uploading its result to the major node in every generation. Wang et al. [108] design an infrastructure which has the ability to leverage all of the available GPU cards across multiple machines to concurrently perform the objective evaluation for a batch of individuals. Note that Colangelo et al. [191] , [192] design a reconfigurable hardware framework that fits the ENAS. As they claimed, this is the first work of conducting NAS and hardware cooptimization.
Furthermore, Lu et al. [101] adopt the concept of a proxy models which are small-scale versions of the intended architectures. For example, in a CNN architecture, the number of layers and the number of channels in each layer are reduced. However, the drawback of this method is obvious: the loss of prediction accuracy. Therefor, they perform an experiment to determine the smallest proxy model that can provide a reliable estimate of performance at a larger scale.
All the above methods obtain the fitness of individuals by directly evaluating the performance on the validation dataset. An alternative way is using the indirect methods, namely the performance predictors. As summarized in [37] , the performance predictors can be divided into two categories: performance predictors based on the learning curve and endto-end performance predictors, both of which are based on the training-predicting computational paradigm. This does not mean the performance predictor does not undergo the training phase at all, while it means learning from the information obtained in the training phase, and use the knowledge learned to make a reasonable prediction for other architectures. Ralwal et al. [34] take the use of the learning curve based predictor where the fitness is not calculated in the last epoch but is predicted by the sequence of fitness from first epochs. Specifically, they use a Long Short Term Memory (LSTM) [193] as a sequence to sequence model, when input the validation perplexity of the first epochs, the output will be the validation perplexity prediction of the subsequent epochs. Sun et al. [37] use an end-to-end performance predictor which does not need any extra information, even the performance of individuals of the first several epochs. Specifically, they adopt a method based on the random forest to accelerate the fitness evaluation in ENAS. When the random forest receives a newly generated architecture as input, adaptive combination of a huge number of regression trees which have been trained in advance in the forest gives the prediction.
This section lists different fields ENAS involved. Actually, the ENAS can be applied to wherever the neural networks can be applied. The following Table V shows the wide range  of applications and Table VI displays the performance of extraordinary ENAS methods on two popular and challenging dataset for image classification, namely CIFAR-10 and CIFAR-100. Both of these two tables can represent what ENAS has achieved so far.
A. Overview Table V shows the applications of ENAS. This is a incomplete statistics but also contains a wide range of applications.
Generally, these fields of application can be grouped into the following five categories:
(1) Image and signal processing, including image classification which is the most popular and competitive field, image to image processing (including image restoration, image denoising, super-resolution and image inpainting), emotion recognition, speech recognition, language modeling and face de-identification.
(2) Biological and biomedical tasks, including medical image segmentation, malignant melanoma detection, sleep heart study and assessment of human sperm.
(3) Predictions and forecasting about all sorts of things, including the prediction of wind speed, car park occupancy, time series data, financial and usable life, the forecasting of electricity demand time series, traffic flow, electricity price and municipal waste.
(4) Machine, including engine vibration prediction, Unmanned Aerial Vehicle (UAV), bearing fault diagnosis and predicting general aviation flight data.
(5) Others, including crack detection of concrete, gammaray detection, multitask learning, identify galaxies, video understanding and comics understanding.
In Table V , it is obviously to see that many ENAS methods are applied on the image classification task. The benchmark dataset, CIFAR-10 which contains a total of ten classes, and the CIFAR-100 is the advanced dataset including a hundred of classes. These two datasets are widely used in image classification tasks, and the accuracy on these two challenging datasets can represent the ability of the architecture. We collect the well-performed ENAS methods tested on these two datasets. The Table VI shows the test results on the two datasets of different state-of-the-art methods separately, where the methods are ranked in ascending order of their best accuracy on CIFAR-10, i.e. the methods are ranked in descending order of their error rate. The column "CIFAR-10" and "CIFAR-100" denote the error rate of each method on the corresponding datasets respectively. Furthermore, the "GPU Days" denotes the total search time of each method, it can be calculated by the Equation 9  TABLE V  APPLICATION   Category  Applications  References  1 Image classification [9] , [19] - [21] , [26] , [29] - [32] , [36] , [38] - [40] , [42] - [44] , [49] , [50] , [56] - [58] , [61] , [62] , [67] , [68] , [70] , [71] , [71] , [74] , [78] , [80] , [83] , [84] , [88] - [90] , [92] , [95] , [96] , [98] , [99] , [101] , [102] , [104] , [105] , [107] - [110] , [112] , [114] , [115] , [118] , [120] , [121] , [123] , [125] , [126] , [128] - [133] , [135] , [137] , [141] , [149] , [156] , [158] , [161] , [162] , [172] , [173] , [175] , [183] , [192] , [194] - [197] 1
Image to image [46] , [94] , [97] , [113] , [119] , [127] , [152] 1
Emotion recognition [91] , [145] 1
Speech recognition [54] , [81] , [103] , [138] , [163] 1
Language modeling [34] , [115] 1
Face De-identification [198] 2
Medical image segmentation [38] , [69] , [72] , [100] , [106] , [116] , [122] 2
Malignant melanoma detection [55] , [ Comics understanding [53] , [201] GP U Days = T he number of GP U s × t
where the t denotes the search time of the method. "Parameters" denotes the total number of parameters which can represent the capability of an architecture and the complexity. Actually, this is not a totally fair comparison. The reason can be summarized in the following two aspects: (1) The encoding space including the initial space and the search space are not the same. There are two extreme cases in initial space: trivial initialization which starts at the simplest architecture and rich initialization which starts at a well-designed architecture (e.g. ResNet-50 [2] ). And the size of search space is largely different, e.g., Ref [44] only takes the kernel size into search space. (2) Different tricks used in the methods, e.g. the "cutout", can make the final results unfair, too. The "cutout" refers to a regularization method [204] used in the training of CNNs, which could improve the final performance.
Anyhow, Table VI shows the progress of ENAS in image classification according to the accuracy on CIFAR-10: Large-Scale Evolution [19] (5.4%, 2017), LEMONADE [105] (2.58% 2018), NSGANet [101] (2.02%, 2019). Many ENAS methods have lower error rate than ResNet-110 [2] with 6.43% error rate on CIFAR-10, which is a manually well-designed architecture. Therefore, the architecture found by ENAS can reach the same level as or exceed the architecture designed by experts. It proves that the ENAS is reliable and can be used in other application field.
Despite the positive results of the existing methods, there are still some challenges and issues which need to be addressed.
The effectiveness of ENAS is questioned by many researches. Wistuba et al. [24] notice that the random search can get a well-performed architecture and has proven to be an extremely strong baseline. Yu et al. [205] show the state-of-the-art NAS algorithms perform similarly to the random policy on average. Liashchynskyi et al. [206] compare grid search, random search, and GA for NAS and the result is that the architecture obtained by GA and the random search have similar performance. There is no need to use the complicate algorithms to guide the search process if the random search can outperform the NAS based on EC paradigm.
However, the population operator in [206] only contains a recombination operator which could not represent the whole effectiveness of ENAS. Although the random search can find a well-performed architecture in experiment, it can not guarantee that it will find a good architecture every time. Moreover, in [99] , [137] , the evolutionary search is more effective than random search. There is an urgent need to design a sophisticated experiment to tell the effectiveness of the state-of-the-art ENAS methods, especially in a large encoding space.
In Section IV, two types of operators are introduced. We note that some methods like Large-Scale NAS [19] only use single individual based operator (mutation) to generate offspring. The main reason why they do not choose the crossover operator in their method come from two aspects: the first is for simplicity [177] , and the second is that simply combining a section of one individual with a section of another individual seems ill-suited to the neural network paradigm [36] . Also, in [24] , the authors believe that there is no indication that a recombination operation applied to two individuals with high fitness would result into an offspring with similar or better fitness.
However, the supplemental materials in [20] demonstrate the effectiveness of the crossover operator in this method. This method can find a good architecture in short order with the help of the crossover. On the contrary, when the crossover is not performed, the architecture found is not really promising, unless it runs for a long time. In fact, the mutation operator let an individual explore the space around itself, and it is a gradually incremental search process like searching step by step. The crossover (recombination) can generate offspring dramatically different from the parents by contrast, which is more like a stride. So, this operator has the ability to efficiently find a promising architecture. Chu et al. [119] prefer that while a crossover mainly contributes to exploitation, a mutation is usually aimed to introduce exploration. These two operators play different role in the evolving progress. But there is not a sufficient explanation how the crossover operator works. Maybe some additional experiments need to be done on the methods that do not include the crossover operator.
Section V-C has introduced the most popular and effective way to reduce the evaluation time. In a nutshell, it can be described as a question that strikes the balance between the time spent and the accuracy of the evaluation. Because of the unbearable full training time, we must compromise as little as we can on evaluation accuracy in exchange for significantly reduction in evaluation time without sufficient computing resources.
Although a lot of ENAS methods have adopted various kinds of ways to shorten the evaluation time and even though Sun et al. [37] specifically proposed a method of acceleration, the research direction of search acceleration is just getting started. The current approaches have many shortcomings that need to be addressed. Furthermore, there is no baseline and common assessment criteria of the search acceleration methods. It is a big challenge to propose a novel kind of method to evaluate the architecture accurately and quickly. Table V shows variable applications which are researched by the ENAS currently. But these are just a small part of all areas of neural network application. Actually, ENAS can be applied wherever neural networks can be applied and automate the process of architecture designed which should have done by experts. Moreover, plenty of the image classification successes of ENAS have proven the ENAS has the ability to replace experts. The automated architecture design is a trend.
However, this process is not totally automated. The encoding space (search space) still needs to be designed by experts for different applications. For example, for the image processing tasks, the CNNs are more suitable, so the encoding space contains the layers including convolution layers, pooling layers and fully connected layers. While for the time series data processing, the RNNs are more suitable, so the encoding space may contain the cells including ∆-RNN cell, LSTM [193] , Gated Recurrent Unit (GRU) [207] , Minimally-Gated Unit (MGU) [208] and Update-Gated RNN (UGRNN) [209] . The two manually determined encoding space already contains a great deal of artificial experience and the components without guaranteed performance are excluded. The problem is: can a method search the corresponding type of neural network for multiple tasks in a large encoding space including all the popular wide used components? Instead of searching one multitask networks [200] which learns several tasks at once with the same neural network, the aim is to find appropriate networks for different tasks in one large encoding space.
Section VI-B gives a brief introduction of the unfair comparison. The unfairness mainly comes from two aspects: (1) the tricks including cutout [204] , ScheduledDropPath [210] , etc. (2) The different encoding space. For aspect (1), we notice some ENAS methods [20] have reported the results with and without the tricks. As aspect (2), the well-designed search space is widely used in different ENAS methods. For instance, the NASNet search space [210] is also used in [130] , [131] because it is well-constructed even that random search can perform well. The comparison under the same condition can tell the effectiveness of different search methods.
Fortunately, the first public benchmark dataset for NAS, the NAS-Bench-101 [211] has been proposed. The dataset contains 432k unique convolutional architectures based on the cell-based encoding space. Each architecture can query the corresponding metrics, including test accuracy, training time, etc., directly in the dataset without the large-scale computation. NAS-Bench-201 [212] is proposed recently and is based on another cell-based encoding space which has no limitations on edges. Compared with NAS-Bench-101, which was only tested on CIFAR-10, this dataset collects the test accuracy on three different image classification datasets (CIFAR-10, CIFAR-100, ImageNet). But the encoding space is relatively small, and only contains 15.6k architectures. Actually, experiment with different ENAS methods on these benchmark datasets can get a fair comparison and it will not take too much time. However, these datasets are only based on the cell-based encoding space and can not contain all the search space of the existing methods, because the other basic units (layers and blocks) are built by more hyper-parameters which may lead to a larger encoding space.
In the future, a common platform for the comparison needs to be built. This platform must have several benchmarks encoding space, such as the NASNet search space, NAS-Bench-101 and NAS-Bench-201. All the ENAS methods can directly test on the platform. Furthermore, this platform also needs to solve the problem that the different kinds of GPUs have different computing power which may cause an inaccurate GPU Days based on the different standards. The GPU Days can not compare directly until they have a common base line of computing power.
This paper provided a comprehensive survey of ENAS. We introduced the ENAS from four aspects: population initialization, population operators, evaluation and selection following the common flowchart which can be seen in Fig. 2 . The various applications and the performance of the state-ofthe-art methods on image classification are also summarized in tables to demonstrate the wide applicability and the promising ability. Challenges and issues are discussed to identify the future research in this field.
To be specific, firstly, the survey shows the different individual presentation and encoding space. We divide the encoding space into two stages: initial space and search space where the former one defines the initial conditions whereas the latter one limits the novel architectures can be found in the evolving process. In addition, the constraints on the encoding space are also introduced by categories. Secondly, the population operators are introduced in two parts, namely the single individual based operator and the multiple individual based operator. A variety of EC algorithms use respective regulation to generate new individuals. Based on the original algorithms, many novel methods have proposed improvement to get a stable and reliable search capability. Thirdly, the evaluation criteria divide the ENAS into two categories: single object and multiple object. Furthermore, we demonstrated the existing methods to shorten the long evaluation time which is a huge obstacle to efficiency.
Despite the state-of-the-art methods have achieved some success, the ENAS still faces challenges and issues. The first important issue is whether the EC-based search strategy is useful. If the result is at the same level of baseline (e.g., random search), it is unnecessary to design the complex operators on the population. A sophisticated experiment is in urgent need to tell the effectiveness, especially in a large encoding space. Secondly, the crossover operator is an undirected multiple individual based operator and there is no sufficient explanation how the crossover operator works. Besides, the ENAS is just beginning a new era, so that there is a lot of uncharted territory to be explored. Moreover, a unified standard or platform is demanded to make a fair comparison.