# Multi-Objective Evolutionary Neural Architecture Search with Weight-Sharing Supernet
## 5 Experiments
### Search Performance Comparison on CIFAR-10 Dataset  
#### 5.1.1 Experimental Settings  
The hyper-parameter settings for the bi-population MOEA/D-based NAS method are shown in Table 1. Specifically, the number of epochs for the warm-up phase is set to 50. The algorithm will update for 45 periods, with 10 epochs of weight optimization in each period. Therefore, for the weight parameters \(\omega\), there are \(50+45\times 10=500\) epochs of training and updating in total, which is set to the same as CARS for fair comparisons. The overall population size is set at 32, which means each sub-population's size is 16. Each individual has four neighbors, and the Tchebycheff aggregation function is adopted when updating  
Figure 6: Evolutionary operators based on the DARTS search space.  
the neighboring individuals. In the inter-population communication stage, the number of exchanged individuals is set to 3. Note that the adopted values of hyper-parameters are tested and chosen based on the experiments on the widely used NAS benchmark datasets NAS-Bench-101 [48] and NAS-Bench-201 [49]. In these benchmark datasets, all the network architectures in the search space are evaluated in advance, and researchers can efficiently design NAS strategies and tune the hyper-parameters in the algorithm.  
To verify the effectiveness of the bi-population MOEA/D-based NAS method, the algorithm is compared with the traditional NSGA-II, the improved NSGA-II (CARS), and the traditional MOEA/D-based NAS. For the CARS algorithm, the population size is set to 32, while the other settings remain the same as in the original work. For the traditional NSGA-II and MOEA/D-based NAS methods, the population size should be set to 16 to simply verify their performance. For NSGA-II, the number of parents is set to 2 in tournament selection, while the crossover and mutation rates are the same as those in Table 1. In MOEA/D, the number of weight vectors is equal to the population size; the other parameters are consistent with those in Table 1, except the traditional MOEA/D does not include communication individuals.  
The other settings are listed as follows: For experiments on CIFAR-10, 8 network cells and an initial channel of 16 are used in the search phase. The training and validation sets contain 25,000 samples each with a batch size of 128. During training, a cosine-annealing-based scheduler is adopted, with an initial learning rate of 0.025. Stochastic gradient descent is employed with a momentum of 0.9 and a weight decay of \(3\times 10^{-4}\). For the final evaluation phase, the network is constructed with 20 cells, and the initial channels are set to 36. The re-training uses a batch size of 96 and runs for 600 epochs. The settings of the scheduler and the stochastic gradient descent are the same as in the search phase. Additionally, cutout and dropout strategies are applied as in DARTS, with a dropout probability of 0.2. An auxiliary loss function with a weight of 0.4 is also introduced. The search phase of NAS is performed on a single NVIDIA GeForce RTX 3090 GPU. The experiments are coded with Python 3.8.10 and PyTorch 1.8.1.