# Efficient Pruning of Detection Transformer in Remote Sensing Using Ant Colony Evolutionary Pruning
## 3 Methodology
### Global Search and Bayesian Pruning
Upon forecasting all unexplored points, the upper confidence bound is employed to designate the subsequent stable point, characterized as follows:  
\[\mathrm{H}^{*}|\mathrm{H}=\mathrm{GP}\left(\mathrm{\mu(X^{*})},\mathrm{\sigma^{ 2}(X^{*})}\right) \tag{11}\]  
\[\mathrm{\mu(X^{*})}=\mathrm{K(X^{*},X)K(X,X)^{-1}H} \tag{12}\]  
\[\mathrm{\sigma^{2}(X^{*})}=\mathrm{K(X^{*},X^{*})-K(X^{*},X)K(X,X)^{-1}K(X,X^{* })} \tag{13}\]  
where K is the covariance matrix and X is the vector of existing observations. After completing the prediction of all unknown points, the upper confidence bound is used to collect the next stable point, which is expressed as follows:  
\[X^{*}\leftarrow\underset{\mathrm{x}\in\mathbb{X}}{\mathrm{argmax}}\mathrm{UCB (X^{*})} \tag{14}\]  
\[\mathrm{UCB(X^{*})}=\mathrm{\mu(X^{*})}+\mathrm{\beta^{1/2}\sigma(X^{*})}, \tag{15}\]  
where \(\mathbb{X}\) denotes the collection of all forecasted points, and \(\mathrm{\beta}\) is the fluctuation coefficient utilized for harmonizing the anticipation and variance. Subsequently, a complete round of global search is performed after the processes of soft pruning, ant evaluation, ant evolution, and fine-tuning. The obtained model will serve as the input for Bayesian optimization, guiding the next round of the search process. Below is a step-by-step explanation of each process. Soft Pruning: After selecting a path, ants execute soft pruning followed by validating the accuracy on the validation set. Soft pruning is simply zeroing out the parameters that need to be pruned without actually removing those parameters from the neural network. This smooth pruning process makes the evaluation process much easier, as it can recover from incorrectly removed weights. Evaluation of ants: After soft pruning, if the model has a high accuracy, it means that the path chosen by the ant can better maintain neural network performance. We use it as an evaluation index for ants. At the end of each round of the search, the evaluation index of all ants will be ranked to distinguish the competitiveness level of the ants. Evolution of ants: Ants exhibiting superior rankings transform into new elite ants, whereas ants with inferior rankings are demoted to regular ants for the next exploration cycle. Preserving the optimal architecture determined in each exploration cycle is facilitated by transferring the best architecture discovered by each ant colony to the first elite ant. This ant will directly opt for the inherited path in the ensuing exploration cycle without exploration. Should all elite ants within a colony regress to regular ants, the non-competitive path will not be retained. This evolutionary strategy has two advantages: one is that the dominant ant individuals can be retained, and the other is that the search range is increased to prevent the colony from falling into a local optimum. Furthermore, owing to inter-colony competition, ants with lower rankings will be eliminated. If all ants perish within a colony, the colony will vanish, abstaining from future explorations. Fine-tuning: Upon concluding the exploration, the optimal architecture uncovered will undergo fine-tuning. The accuracy and quantity of model parameters after fine-tuning will be integrated back into Equation (8), and the Bayesian optimizer will refine the Gaussian process. Subsequently, a new pruning ratio vector will be provided by the Bayesian optimizer, and the exploration process will persist in a cyclical manner until the termination criterion is satisfied.