# Multi-Objective Evolutionary Neural Architecture Search with Weight-Sharing Supernet
## 5 Experiments
### Search Performance Comparison on CIFAR-10 Dataset
#### 5.1.2 Experimental Results  
Figure 7 illustrates the search phase of the multi-objective evolutionary NAS methods based on the proposed framework for CIFAR-10. Specifically, the fitness values of the population are recorded and visualized in different colors every five periods. The population after the last period is visualized in red. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Parameters** & **Description** & **Value** \\ \hline \(E_{\text{warmup}}\) & Number of warm-up epochs & 50 \\ \(N_{\text{period}}\) & Number of update periods & 45 \\ \(E_{\omega}\) & Number of weight optimization epochs per period & 10 \\ \(N_{\text{pop}}\) & Population size & 32 \\ \(N_{\text{w}}\) & Number of weight vectors & 16 \\ \(T\) & Number of neighbours & 4 \\ \(g(\cdot)\) & Aggregation function & \(g^{te}\) \\ \(pc\) & Crossover rate & 0.5 \\ \(pm\) & Mutation rate & 0.1 \\ \(N_{\text{comm}}\) & Number of exchanged individuals & 3 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: The hyper-parameter settings for the bi-population MOEA/D-based NAS algorithm. Firstly, it can be observed from Figure 7a,c that applying traditional NSGA-II and MOEA/D algorithms under the proposed framework will lead to bad results. Both algorithms have the small model trap problem, in which the populations prematurely converge to the region of small models with low error rates (with model sizes around 0.12-0.16 M) in the early stages, lacking sufficient exploration of the large models. Subsequent experiments also verify that these small models, which perform well during the search phase, show decreased validation performance on complex datasets. Figure 7: Visualization of the search phase of the multi-objective evolutionary NAS methods for CIFAR-10. (**a**) Based on traditional NSGA-II; (**b**) based on improved NSGA-II (CARS); (**c**) based on traditional MOEA/D; (**d**) based on bi-population MOEA/D (without communication); (**e**) based on bi-population MOEA/D (with communication, visualized by the global population); (**f**) based on bi-population MOEA/D (with communication, visualized by the sub-populations). Further comparison between the NAS method based on the traditional NSGA-II and CARS algorithms (Figure 6(b)) reveals that the CARS algorithm can balance the search for models with different sizes and maintain good diversity in the population. On the other hand, by observing the Pareto front obtained by CARS, we can preliminarily infer the distribution of network architectures in the DARTS search space. After the model size increases to 0.2 M, the lowest error rate of the models hardly decreases. In this case, the algorithms may ignore the large models with performance similar to the small ones, but these large models may achieve better performance than the small ones on test data. Therefore, in the DARTS search space, the NAS algorithms should intentionally retain some of the well-performing large models. Figure 6(d) illustrates the search process of the bi-population MOEA/D NAS method without the communication stage. Compared to Figure 6(c), it can be observed that the population exhibits improved exploration in the entire search space. However, the two sub-populations update independently and lack the necessary interaction. As a result, the global population is optimized in two extreme directions, and models with medium sizes (with model sizes around 0.25-0.35 M) are neglected. To address this issue, the communication stage is appended to the bi-population MOEA/D algorithm. Figure 6(e) shows the optimization process for the global population. Compared to the one without communication, the bi-population MOEA/D algorithm with communication exhibits a broad exploration range and maintains good diversity in population, achieving results similar to CARS. Figure 6(f) further visualizes the evolution of the two sub-populations separately, with \(A_{1}\) represented by dots and \(A_{2}\) by triangles.