# Multi-Objective Evolutionary Neural Architecture Search with Weight-Sharing Supernet
## 5 Experiments
### Test Performance Comparison on CIFAR-10 Dataset
#### 5.2.2 Experimental Results  
The test performance of the network architectures searched by various NAS methods on the CIFAR-10 dataset is summarized in Table 2. Note that the multi-objective methods are labeled with the abbreviation MO. Additionally, the proposed bi-population MOEA/D-based NAS method is abbreviated as Bi-MOEA/D-NAS, and four well-distributed models are selected to evaluate the test performance.  
Overall, it can be observed that the majority of network architectures searched by NAS methods can achieve higher accuracy with smaller model sizes compared to the manually designed DenseNet-BC, which validates the practicability of NAS. Compared with the reinforcement-learning-based NAS methods, Bi-MOEA/D-NAS can discover models with fewer parameters and higher test accuracy than ENAS. Although it achieves slightly lower accuracy compared to NASNet-A with similar model sizes (Bi-MOEA/D-NAS-4), NASNet has higher computational costs (2000 GPU days) than Bi-MOEA/D-NAS (0.5 GPU days). In comparison to the gradient-based DARTS, Bi-MOEA/D-NAS can obtain better network architectures (DARTS-V2 vs. Bi-MOEA/D-NAS-4) in similar model sizes.  
Then, we focus on the comparison between various EC-NAS algorithms. AmoebNet-A and hierarchical evolution, which were proposed early, generally require substantial computational resources to search for the optimal network architectures. In contrast, NSGA-Net, LEMONADE, CARS, and the proposed Bi-MOEA/D-NAS algorithms can improve search efficiency while maintaining search performance. In addition, these methods can search for network architectures of different sizes. Compared to NSGA-Net and LEMON-ADE, CARS and Bi-MOEA/D-NAS can obtain more diverse architectures in a shorter search time. Comparing CARS with Bi-MOEA/D-NAS, both methods can complete the search process within half a day. In terms of the larger models, Bi-MOEA/D-NAS has a slightly lower performance than CARS (CARS-H vs. Bi-MOEA/D-NAS-4), but it can achieve better performance on the smaller models with similar sizes (CARS-B and CARS-C vs. Bi-MOEA/D-NAS-1 and Bi-MOEA/D-NAS-2).  
Therefore, the above experiment demonstrates the potential of EC in solving NAS problems. Furthermore, the results validate the effectiveness and practicality of the proposed Bi-MOEA/D-NAS method. Compared to the other representative NAS method, Bi-MOEA/D-NAS can efficiently excavate diverse models with high accuracy in a short search time.  
\begin{table}
\begin{tabular}{l c c c c} \hline \hline \multirow{2}{*}{**Architecture**} & \multirow{2}{*}{**Test Error (\%)**} & \multirow{2}{*}{**\#Params (M)**} & \multicolumn{2}{c}{**Search Cost**} & \multirow{2}{*}{**Search Method**} \\  & & & **(GPU Days)** & \\ \hline DenseNet-BC & 3.46 & 25.6 & - & Manual design \\ ENAS & 2.91 & 4.2 & 4 & Reinforcement learning \\ NASNet-A & 2.65 & 3.3 & 2000 & Reinforcement learning \\ DARTS-V1 & 3.00 & 3.3 & 1.5 & Gradient based \\ DARTS-V2 & 2.76 & 3.3 & 4 & Gradient based \\ AmoebaNet-A & 3.12 & 3.1 & 3150 & Evolution \\ Hierarchical evolution & 3.75 & 15.7 & 300 & Evolution \\ NSGA-Net-1 & 3.85 & 3.3 & 8 & Evolution (MO) \\ NSGA-Net-2 & 2.50 & 26.8 & 4 & Evolution (MO) \\ LEMONADE-1 & 3.69 & 1.1 & 80 & Evolution (MO) \\ LEMONADE-2 & 3.05 & 4.7 & 80 & Evolution (MO) \\ LEMONADE-3 & 2.58 & 13.1 & 80 & Evolution (MO) \\ CARS-B & 2.87 & 2.7 & 0.4 & Evolution (MO) \\ CARS-C & 2.84 & 2.8 & 0.4 & Evolution (MO) \\ CARS-E & 2.86 & 3.0 & 0.4 & Evolution (MO) \\ CARS-H & 2.66 & 3.3 & 0.4 & Evolution (MO) \\ Bi-MOEA/D-NAS-1 & 2.80 & 2.53 & 0.5 & Evolution (MO) \\ Bi-MOEA/D-NAS-2 & 2.86 & 2.71 & 0.5 & Evolution (MO) \\ Bi-MOEA/D-NAS-3 & 2.85 & 2.88 & 0.5 & Evolution (MO) \\ Bi-MOEA/D-NAS-4 & 2.72 & 3.26 & 0.5 & Evolution (MO) \\ \hline \hline \end{tabular}
\end{table}
Table 2: Test performance of the searched network architectures on CIFAR-10.