# Multi-Objective Evolutionary Neural Architecture Search with Weight-Sharing Supernet
## 2 Related Works
### Evolutionary-Computation-Based Neural Architecture Search
#### 2.3.2 Population Updating Strategy  
After initialization, the population continuously evolves, eliminates the worst individuals, and generates new ones, which is generally controlled by a specific population updating strategy. In EC-NAS, the most commonly used methods for population updating are evolutionary algorithms.  
In evolutionary algorithms, the key operators involve selection, crossover, and mutation. Specifically, selection operators are mainly used for mating selection and environmental selection. Elitism is one of the most widely used methods that directly selects the best individuals in the population, aiming to preserve the best genes as much as possible. However, it may result in premature convergence. Alternatively, RegularizedEvo [12] employs an aging strategy to update the architecture population. In each iteration, a subset of parents is randomly selected, and the best one is chosen for mutation to generate a new offspring, while the oldest individual in the population is eliminated. Some algorithms tend to preserve individuals with more diversity. For example, [PERSON] et al. [27] selected the parents based on their distance from and difference to the other individuals, similar to the crowding distance in NSGA-II [28].  
After selecting parent individuals, crossover and mutation operators can be applied to generate offspring individuals. For the crossover operation, the simplest approach is to use a single-point crossover operator [29], where a segment of the chromosomes of two individuals is exchanged starting from a specific position. Although this method is intuitive and easy to implement, it may disrupt the connectivity of the network architecture in the context of NAS. Another commonly used crossover operator is simulated binary crossover [30], which can be applied in real-valued encoding schemes. On the other hand, mutation operations aim to optimize the population in a more exploratory direction. In discrete encoding schemes, the value on a specific gene can be mutated to another candidate choice, which may represent the replacement of the operation on a specific edge or change the target node of the chosen edge to another. As for continuous encoding, polynomial mutation or Gaussian mutation methods can be used [31; 32].  
The strategies described above mostly focus on a single objective when evaluating the neural networks, for example, the classification accuracy of image classification tasks. However, in multi-objective EC-NAS [10; 13; 14; 33], more than one metric is considered simultaneously to optimize the networks. These metrics, such as model accuracy and the number of model parameters, are always in conflict with each other. For instance, a network with high accuracy typically has a complex architecture, while a resource-constrained device like a mobile phone makes it hard to afford such computational costs, and the model performance must be sacrificed to some extent. The simplest way to solve multi-objective EC-NAS is to transform the multi-objective problem into a single-objective problem by taking the weighted sum of the given objectives. The other way is to use multi-objectiveevolutionary algorithms, such as the NSGA-II [28] and MOEA/D [34] algorithms. This kind of method aims to find a Pareto front in the objective space, which can be a good trade-off between multiple conflicting objectives. For example, LEMONADE [13] treats the objective functions as expensive ones and simple ones. This work is devoted to enhancing search efficiency by increasing the evaluation of low-cost objectives and reducing the evaluation of expensive ones. In CARS [14], the researchers found that the traditional NSGA algorithm only tends to search for small models with high accuracy, so they designed a two-stage non-dominated sorting strategy to balance the search for models with different sizes.