# Multi-Objective Evolutionary Neural Architecture Search with Weight-Sharing Supernet
## 2 Related Works
### Evolutionary-Computation-Based Neural Architecture Search
#### 2.3.3 Population Evaluation Strategy  
Since EC-NAS is a population-based search method, the evaluation of a large number of individuals in the population has become a bottleneck for EC-NAS. Thus, recent studies have made efforts to design useful strategies to accelerate the evaluation process. Some studies employ knowledge inheritance [35] or network morphism strategies. This kind of method considers that the newly generated offspring may retain some of the same structures as their parents, which allows them to inherit the corresponding network weights. In other words, it is possible for newly generated models to preserve parts of the weight parameters from the trained models to achieve a warm start step [11; 18; 32; 36]. Rather than training the network models from scratch, these methods can substantially accelerate the evaluation process. Many recent studies [14; 21; 37] have adopted the idea of the one-shot model [38]. Essentially, the one-shot model is a supemet that contains all the possible nodes and the edge operations of neural networks in the whole search space. A subnet of the supernet represents a candidate network architecture, which is obtained by masking on the supernet, i.e., selecting some of the nodes and edges to form a valid subnet. During the training process, subnets can directly share the weight parameters with the supernet rather than training from scratch. In this way, the calculation cost is greatly reduced. For example, in DARTS, each edge between the feature map nodes in the supemet includes eight optional operations (such as convolutional layers, pooling layers, etc.). Each operation is associated with a weight to indicate its importance. The weight parameters in these eight layers are optimized simultaneously during the training process. Then, the operation with the highest weight will be selected from each node to form a candidate network when evaluating the real performance. In addition, strategies with surrogate models tend to replace the expensive performance evaluation with a performance predictor. Specifically, the performance predictor is first trained on the training set to capture some useful information from the training data. Then, it utilizes the learned knowledge to make performance predictions for the other unseen network architectures [39; 40]. In addition, [PERSON] et al. [41] proposed a learning curve predictor in which the fitness of individuals in the final iteration is predicted based on their performance in previous iterations. By employing surrogates or predictors, the search efficiency of EC-NAS can also be promoted. Other types of strategies focus on reducing the fitness evaluation times of individuals during the evolutionary process. For instance, [PERSON] et al. [42] designed Long-Term Memory Assistance (LTMA) to store information throughout the entire evolutionary process. Once an identical individual that has appeared in the history is searched, its fitness value can be directly retrieved from the record, and repeated evaluation is avoided. In this way, computational resources can be considerably saved for further exploration and exploitation in the search space. 3 Multi-Objective Evolutionary Neural Architecture Search Framework Based on Weight-Sharing Supernet  
As described above, EC-NAS methods are characterized by their strong global search capability but are limited by their inefficiency in evaluating a large number of individuals in the architecture population. On the other hand, gradient-based NAS often utilizes one-shot models and the weight-sharing strategy to train on a supernet, which can effectively reduce the search time. However, this type of method heavily relies on the correct optimization of gradients, making it difficult to design. Additionally, gradient-based NAS generally outputs network architectures oriented towards a single objective, like classification accuracy, lacking diversity and consideration for the other metrics of neural networks.