# Multi-Objective Evolutionary Neural Architecture Search with Weight-Sharing Supernet
###### Abstract  
Deep neural networks have played a crucial role in the field of deep learning, achieving significant success in practical applications. The architecture of neural networks is key to their performance. In the past few years, these architectures have been manually designed by experts with rich domain knowledge. Additionally, the optimal neural network architecture can vary depending on specific tasks and data distributions. Neural Architecture Search (NAS) is a class of techniques aimed at automatically searching for and designing neural network architectures according to the given tasks and data. Specifically, evolutionary-computation-based NAS methods are known for their strong global search capability and have aroused widespread interest in recent years. Although evolutionary-computation-based NAS has achieved success in a wide range of research and applications, it still faces bottlenecks in training and evaluating a large number of individuals during optimization. In this study, we first devise a multi-objective evolutionary NAS framework based on a weight-sharing supernet to improve the search efficiency of traditional evolutionary-computation-based NAS. This framework combines the population optimization characteristic of evolutionary algorithms with the weight-sharing ideas in one-shot models. We then design a bi-population MOEA/D algorithm based on the proposed framework to effectively solve the NAS problem. By constructing two sub-populations with different optimization objectives, the algorithm can effectively explore network architectures of various sizes in complex search spaces. An inter-population communication mechanism further enhances the algorithm's exploratory capability, enabling it to find network architectures with uniform distribution and high diversity. Finally, we conduct performance comparison experiments on image classification datasets of different scales and complexities. Experimental results demonstrate the effectiveness of the proposed multi-objective evolutionary NAS framework and the practicality and transferability of the introduced bi-population MOEA/D-based NAS method compared to existing state-of-the-art NAS methods. deep learning; neural architecture search; multi-objective evolutionary algorithms; image classification +
Footnote †: journal: Applied Sciences  
0000-0001-5000-0000-0000-0000-0000-0000-00000-0000-00000-00000-0000-0000-00000-0000-00000-00000-00000-00000-0000-00000-00000-00000-00000-00000-00000-00000-00  
high-performance neural networks were usually designed by professional experts manually, such as the well-known ResNet [3] and DenseNet [4]. However, in many application scenarios, researchers lack sufficient domain knowledge to design such sophisticated network architectures. They are easily constrained by the existing network architecture patterns and find it difficult to design more innovative neural networks. On the other hand, the design of neural network architectures depends on specific tasks and relevant data to some extent. The optimal neural network architecture may vary as the data distribution changes. In such cases, the cost of manually tuning and using trial and error on neural networks becomes relatively high. Based on these challenges, researchers now mainly focus on developing algorithms to automatically design optimal neural network architectures. Neural Architecture Search (NAS) is a method aimed at automatically searching for and designing optimal neural network architectures based on specific applications and data. It has drawn continuous attention in recent years due to its convenience, flexibility, and effectiveness. To date, NAS methods have obtained neural networks that outperform manually designed ones in various deep learning tasks. Essentially, NAS tries to optimize the hyper-parameters related to neural network architectures, with different optimization objectives for different types of neural networks.