# NDARTS: A Differentiable Architecture Search Based on the Neumann Series
## 3 Experiment
### Simulation Experiment
#### 3.1.5 Performance Experiment  
The ablation experiment determined the optimal parameters of NDARTS. Subsequently, we tested the performance of NDARTS in the DARTS search space and NAS-Bench-201 search space under the optimal parameters and compared it with other algorithms at similar model scales. Because of the limited computational resources, we mainly tested gradient-based algorithms, and other results are compared using the results of the original paper, marked with * in Tables. The algorithm tested in this article will provide a reference for the accuracy of the validation set as an additional condition, and the program list will be provided in Appendix A.  
DARTS search space. Comparing the results of the NDARTS algorithm in the DARTS search space with other NAS algorithms, the results on the CIFAR-10, CIFAR-100, and ImageNet datasets are shown in Tables 1-3, respectively. Overall, compared with other types of methods, such as random search and RL, NDARTS has a significant increase in performance. And methods based on random search, RL, and EA have longer computational time, and NDARTS can achieve better performance in a short period. Although the PNAS [9] algorithm based on sequential models also requires smaller computational resources, the cost is to reduce the accuracy of the model. Compared with the baseline algorithm DARTS, which is based on gradient descent, NDARTS improves the performance of the algorithm while reducing a certain computational cost. FairDARTS [38] relaxes the choice of operations to be collaborative, where the authors let each operation have an equal opportunity to develop its strength, but our algorithm performed better than FairDARTS. PDARTS [24] searches for neural network architecture progressively, achieving good accuracy at a higher computational cost, while NDARTS achieves better performance at a lower computational cost. PC-DARTS [25] reduces computational costs by sampling channels, and can quickly obtain models with better performance. Although NDARTS has a slightly slower speed than PC-DARTS, it has better performance. In the DARTS search space, NDARTS achieved optimal performance on all three datasets. The results on the CIFAR-10 dataset are shown in Table 1. This article ran some algorithms and displayed the accuracy of the test set in Figure 12 (truncated to 50 epochs due to different algorithm settings). It can be seen that the performance of models searched based on gradient methods is generally better than those searched by algorithms based on random search, RL, and other methods. Among gradient-based methods, NDARTS achieved the best performance on the CIFAR-10 dataset, with a test set error of only 2.37%, which is superior to other gradient methods such as FairDARTS, PDARTS, PC-DARTS, and DARTS with 2.54%, 2.50%, 2.57%, and 2.76%. And, it can be seen from the iteration curve \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Algorithm** & **Val Error (\%)** & **Test Error (\%)** & **Param (m)** & **Search Space** & **Search Method** \\ \hline RandomNAS [39]* & — & 2.65 & 4.3 & DARTS & Random \\ AmoebaNet* & — & 3.34 & 5.1 & NASNet & EA \\ NASNet* & — & 2.65 & 5.3 & NASNet & RL \\ PNAS* & — & 3.41 & 5.1 & DARTS & Sequential \\ ENAS* & — & 2.89 & 4.6 & NASNet & RL \\ \hline GDAS* [40] & — & 2.93 & 3.4 & DARTS & Gradient \\ SETN* [41] & — & 2.69 & 4.6 & DARTS & Gradient \\ FairDARTS [38] & 3.82 & 2.54 & 2.83 & DARTS & Gradient \\ PDARTS & 3.99 & 2.50 & 3.4 & DARTS & Gradient \\ PC-DARTS & 3.88 & 2.57 & 3.6 & DARTS & Gradient \\ DARTS & 4.38 & 2.76 & 3.4 & DARTS & Gradient \\ \hline NDARTS & 3.98 & 2.37 & 3.8 & DARTS & Gradient \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Results of NDARTS (based on DARTS search space) and other NAS algorithms on the CIFAR-10 dataset. in Figure 12 that even if the initial state is worse, NDARTS can still quickly optimize to a better model.