# NDARTS: A Differentiable Architecture Search Based on the Neumann Series
## 3 Experiment
### Simulation Experiment
#### 3.1.4 Ablation Experiment  
We first conducted ablation experiments on the NAS-Bench-201 search space to investigate the impact of parameters \(T,K,\gamma,\gamma_{\alpha}\) on the performance of NDARTS and determine the  
Figure 4: Cell structure of darts search space.  
Figure 5: Cell structure of NAS-Bench-201 search space.  
optimal experimental parameters. The ablation experiment used CIFAR-10 as the training dataset, and a pre-trained model was used to reduce computational complexity. The effects of different parameters were analyzed through 30 epochs of results.  
1. T.  
Parameter T is the number of steps for updating the weight parameter \(\omega\) during the update interval of the architecture parameter \(\alpha\). In theory, the larger the \(T\), the more steps the \(\omega\) updates during each time \(\alpha\) is updating, and the \(\omega\) is closer to the optimal value \(\omega^{*}\). The better the performance of \(\omega\), the better the performance of the super-network when evaluating the architecture, and the more accurate the evaluation results of the subnetwork architecture, which helps the algorithm achieve a more approximate super-gradient estimation. However, the computational cost of the algorithm also increases with an increase in T. According to the experimental results, \(T=4\) achieved a good balance between computational cost and model performance.  
The experimental results indicate that as \(T\) increases, the performance of the algorithm gradually improves. When \(T=1\), both NDARTS and DARTS only optimize the weight parameter \(\omega\) once within the update interval of the architecture parameter \(\alpha\), with the only difference being that NDARTS uses \(\
abla_{a}L_{2}(\omega-\xi\
abla_{\omega}L_{1}(\omega(\alpha),\alpha))\), while DARTS uses \(\
abla_{a}L_{2}^{\prime}(\omega(\alpha),\alpha)\) to update the parameter \(\alpha\)  
As shown in Figure 10, when \(\gamma=0.001\) and \(0.005\), the algorithm can maintain a good accuracy and stability. When \(\gamma=0.01\), the algorithm performance slightly decreases, and the accuracy decreases from 93.58% to 92.02%, indicating good stability. When \(\gamma=0.025\), the accuracy of the algorithm is 92.17%, but the stability of the model begins to decrease. When \(\gamma=0.05\), the performance of the algorithm decreases again, with a model accuracy of 91.22%, and the stability of the algorithm is significantly reduced.  
In NDARTS, the parameter \(\gamma_{\alpha}\) should take a small value to make the coefficient \((1-P)-\frac{L_{\
abla\alpha,\alpha}}{2}\gamma_{\alpha_{m}}(1+D)>0\). As shown in Figure 11, when \(\gamma_{\alpha}\) increases from 0.0001 to 0.001. The performance of the algorithm decreases from 93.58% to 92.04%, with relatively small stability changes. But when \(\gamma_{\alpha}\) increases from 0.001 to 0.005 and 0.01, the performance and stability of the algorithm are significantly reduced.  
In summary, NDARTS has strong sensitivity to parameters \(\gamma\) and \(\gamma_{\alpha}\), chooses smaller \(\gamma\), and \(\gamma_{\alpha}\) can help maintain the performance and stability of NDARTS. The dependence of NDARTS on parameters \(T\) and \(K\) is relatively small. Larger \(T\) and \(K\) can improve the convergence speed and performance of the algorithm, but also increase computational costs. Choosing the appropriate \(T\) and \(K\) can reduce computational costs while maintaining good performance of the NDARTS.  
Figure 11: The impact of \(\gamma_{\alpha}\) on NDARTS.  
Figure 10: The impact of \(\gamma\) on NDARTS.