# Multi-Objective Evolutionary Neural Architecture Search with Weight-Sharing Supernet
###### Abstract
For example, for Convolutional Neural Networks (CNNs), NAS optimizes the architecture parameters such as the number of hidden layers, the number of neurons per layer, filter size, and convolution types. For Recurrent Neural Networks (RNNs), the search may focus on the number of hidden layers, the number of neurons per layer, and the number of time steps [8]. Specifically, the overall framework of NAS mainly consists of three key components: search space, search strategy, and performance estimation strategy [9]. The search space defines the scope within which the NAS algorithms search, referring to a collection of candidate network architectures. It plays a crucial role in the performance and efficiency of the searched network architectures. Existing studies typically combine some prior domain knowledge to restrict the search space, which reduces the complexity of the search process and improves efficiency relatively. Next, the search strategy represents the specific algorithms used to explore and develop the optimal network architectures in the search space. A well-designed search strategy can effectively find promising architectures within a vast search space. Then, the network architectures obtained by the search strategy are usually evaluated on the validation data based on a certain performance estimation strategy, which further guides the optimization direction of the search strategy. In recent years, due to the outstanding performance of evolutionary computation (EC) in solving complex optimization problems, an increasing number of researchers have started to design evolutionary-computation-based NAS methods (also known as EC-NAS) [10; 11; 12; 13; 14]. However, since EC-NAS methods are population-based search methods, they need to evaluate the fitness of each individual in the population during the search process. In other words, the performance of each network architecture represented by an individual must be estimated in each evolving iteration. An intuitive performance estimation strategy is to train the searched networks from scratch and evaluate their classification accuracy on validation data. However, due to the large number of individuals in the evolutionary population, the evaluation step typically consumes a significant amount of time, limiting its scalability and practicality. Therefore, current EC-NAS studies mainly focus on designing effective and efficient search strategies and performance estimation strategies to improve the performance of the searched architectures while reducing computational costs. In this study, we focus on developing a novel EC-NAS method that achieves a trade-off between search effectiveness and efficiency. Specifically, we first design a multi-objective evolutionary NAS framework based on a weight-sharing supernet to effectively reduce the search cost of the population evaluation phase in the traditional EC-NAS method. Additionally, we propose an algorithm based on bi-population MOEA/D to address the potential small model trap problem during the optimization step. This algorithm can explore diverse and high-performance neural network architectures in the large search space. The contributions of this work are summarized as follows:In this study, we design a multi-objective evolutionary NAS framework based on a weight-sharing supernet. This framework combines the population optimization strategy from evolutionary algorithms with the weight-sharing concept of one-shot models to alternately optimize the weight parameters and the architecture parameters in neural networks. It can explore promising network architectures in complex and large search spaces with high efficiency. 2. We propose a bi-population MOEA/D algorithm within the above EC-NAS framework. By setting up two sub-populations to solve different optimization tasks and incorporating a communication mechanism between the sub-populations, the algorithm can thoroughly explore different regions of the search space and obtain uniformly distributed network architectures with high performance and diversity. 3.