## 3 DG-Darts
### DG-DARTS Method  
Figure 1 and Algorithm 2 show the process of DG-DART developed in this study. DG-DARTS has parameters and search space that are the same as those of DARTS. Applying Algorithm 2, the top-\(k\) operations with the largest architecture parameter are selected, and then, the final cells are decided. In this study, \(k=1\), as shown in Figure 1. Another view of the DG-DARTS algorithm is shown in Figure 2. The following are the primary distinctions between Algorithm 1 of DARTS and Algorithm 2 of DG-DARTS:  
* DARTS does not use separate stages, whereas DG-DARTS has two stages.
* Operations in DG-DARTS are clustered between stages 1 and 2, whereas DARTS does not use the concept of clustering.
* In DG-DARTS, operation search space is newly created for stage 2 by clustering results.  
In DG-DARTS, total epochs are divided by two, and each half of the total epochs is separated into two stages.  
```
1:Create operation search space \(\mathcal{O}\)
2:Determine number of epochs to be processed (epochs = 50 in this experiment)
3:initial epoch = 0
4:Stage 1 by using \(\mathcal{O}\)
5:while epoch \(<\) epochs\(/2\)do
6: Update architecture \(\alpha\) by descending \(\
abla_{\alpha}\mathcal{L}_{val}(w,\alpha)\)
7: Update weights \(w\) by descending \(\
abla_{w}\mathcal{L}_{train}(w,\alpha)\)
8: epoch = epoch + 1
9:endwhile
10: Cluster by criteria of gradient of architecture parameter \(\alpha\) over training epochs
11: Sum each \(\alpha\) of clusters
12: Select one-half of the operations from \(\mathcal{O}\), prioritizing the cluster with a larger sum of \(\alpha\)
13: Create a new operation search space \(\mathcal{O}^{\prime}\) by selecting \(|\mathcal{O}|/2\) operations in \(\mathcal{O}\)
14:Stage 2 by using \(\mathcal{O}^{\prime}\)
15:while epoch \(<\) epochsdo
16: Update architecture \(\alpha\) by descending \(\
abla_{\alpha}\mathcal{L}_{val}(w,\alpha)\)
17: Update weights \(w\) by descending \(\
abla_{w}\mathcal{L}_{train}(w,\alpha)\)
18: epoch = epoch + 1
19:endwhile
20: Derive the final architecture based on the learned \(\alpha\)
```  
**Algorithm 2** Process of DG-DARTS  
An example of a vote dispersion problem and its resolution by DG-DARTS can be addressed as follows: We suppose a case in which an edge between nodes 1 and 2 is a convolutional operation that minimizes the loss function. In this case, as the training is processed, the architecture parameter ratio of the convolutional operation is increased by the SoftMax function, as described in Equation (2), while the remaining operations are decreased.  
\[f_{ij}(x_{i})=\sum_{o\in\mathcal{O}_{ij}}\frac{exp(\alpha_{o}^{(i,j)})}{\sum_ {o^{\prime}\in\mathcal{O}}exp(\alpha_{o^{\prime}}^{(i,j)})}o(x_{i}) \tag{2}\]  
where \(x_{i}\) is the node output and \(\alpha^{i,j}\) is the architecture parameter.  
If there are four types of convolution operations, each of the four operations shares the largest weight; thus, the weight of each operation is eventually lower than those of the other operations. Therefore, other operations, such as skip connections, can be selected as the final operation.  
Figure 1: Process of architecture search with multiple search stages.  
Such problems are critical once operations in the search space of DARTS [3] are added or changed. For example, once a good and meaningful operation that was found in another study is added to the existing search space, the newly added operation can have its own shares of weights in the existing search space. If DARTS is applied to a new dataset, a new search space is also composed, then the ratio of the operation group needs to be tuned by analyzing the relations of each operation; otherwise, a vote dispersion problem may occur. To resolve the vote dispersion problem, DG-DARTS was constructed in this study.