## 5 Methodology
### Computation-Aware Searching  
Based on the pruned super neural network, we propose a computation-aware searching process. Generally, we leverage two schemes. The first one is channel pruning, and the second one is a synthetic computation-aware loss function. The architecture search process that integrates channel pruning is shown in Figure 8. Generally, the search process flows into the differentiable architecture search. The difference is that channel pruning could be carried out in several epochs (lines 6-11). Line 6 is a test that checks whether it is the right time to prune the channels. To pass the test, two conditions need to be satisfied. The first one focuses on the gap between training and validation precision. When this gap is greater than a predefined threshold \(T_{1}\), we consider the current overfitting to be noteworthy. Note that, based on this gap, simply pruning a channel is hasty. In the early epochs, because the status is not stable, a large gap could appear between the training and validation precision. To carry out channel pruning in the early stages where the epochs are not stable is not reasonable. Therefore, we add the second condition that requires that the validation precision is greater than another predefined threshold \(T_{2}\). This condition ensures channel pruning can only be carried out during a stable status. When the test in line 6 is passed, Darts_Tiny cuts off half of the channels (line 7) to restrain overfitting. Note that channel shrinkage is not a differentiable operation, e.g., additional training steps are performed to adjust the neural network parameters (line 9) until the validation precision recovers to the third predefined threshold \(T_{3}\) (line 8). Figure 8: The algorithm steps of Darts_Tiny. Figure 7: Darts_Tiny. The super neural network is learned once the architecture search finishes. The termination condition in line 3 could be set to the number of searching epochs, training precision or validation precision. The method of deriving a neural network from the super network mainly follows the settings introduced in Section 4.1. The difference is that the derived neural network follows the main structure of the super network (depicted in Figure 7) in such a way that only three cells are stacked, i.e., the neural network generated has one stem, then three cells and a detection head. Channel pruning decreases the number of computations, which not only restrains overfitting but also improves computational efficiency. In addition to channel pruning, we also leverage a computation-aware loss function to enhance computation efficiency. The computation-aware loss function used is represented by the Formula (8). We can see that _Loss_ is the summation of two parts, where _Loss_ce_ is the cross-entropy loss, and _Ops_ is the total number of multiply-and-accumulate calculations involved. For these two parts, _Loss_ce_ is related to inference accuracy, and _Ops_ is related to computation efficiency. _Ops_ratio_ is the parameter used to balance the two parts in the loss function. To decrease _Loss_, both _Loss_ce_ and _Ops_ should be decreased such that the target neural network searched is expected to perform well in both inference accuracy and computation efficiency. \[Loss=(1-ops\_ratio)\times Loss\_ce+ops\_ratio\times log(Ops) \tag{8}\]  
_Ops_ is obtained by adding the number of multiply-and-accumulate calculations used in different operations. Formulas (9)-(12) provides our method of counting the number of multiply-and-accumulate calculations. \(Ops_{pooling\_3\times 3}\), \(Ops_{depth\_wise}\), \(Ops_{point\_wise}\) and \(Ops_{all\_conv}\), respectively, represent the total multiply-and-accumulate calculations of \(pooling\_3\times 3\) (\(avg\_pool\_3\times 3\) and \(max\_pool\_3\times 3\) ), depth-wise convolution, point-wise convolution and dilated convolution.