# Efficient Pruning of Detection Transformer in Remote Sensing Using Ant Colony Evolutionary Pruning
## 3 Methodology
### Pre-Search  
This stage encompasses the initial exploration conducted by each ant colony within their distinct modules, with the objective of enhancing the diversity of the discovered architectures. During the exploration phase of the ant colony, each individual ant is initially allocated a stochastic exploration factor denoted as \(\alpha\). Ants possessing higher \(\alpha\) values will prioritize uncharted pathways, whereas those with lower \(\alpha\) values will prioritize pathways exhibiting elevated pheromone concentrations. In the absence of a reference pheromone concentration in the inaugural search round, the L1 norm of weights is employed as the preliminary environmental pheromone level. In this preliminary search stage, the ant colony will emphasize exploration; thus, ants are endowed with a relatively elevated exploration factor at the initialization phase. Subsequently, the stage of ant path selection ensues, requiring a delicate equilibrium between exploitation and exploration, simulating the behavior of real-world ants. Prior to embarking on a step, an ant must determine whether to exhibit exploitative or exploratory behavior. \[\mathrm{A}_{\mathrm{k}}^{\mathrm{(i)}}= \begin{cases}1,&\text{if }\mathrm{\theta}_{ \mathrm{k}}^{\mathrm{(i)}}<\alpha\\ 0,&\text{otherwise}\end{cases} , \tag{6}\]  
where \(\mathrm{A}_{\mathrm{k}}^{\mathrm{(i)}}\) denotes the behavior of the i-th pathway in the k-th stratum, while 0 signifies exploitation and 1 signifies exploration. \(\mathrm{\theta}_{\mathrm{k}}^{\mathrm{(i)}}\) signifies a decision-making auxiliary parameter, a unique random assignment to each channel preceding behavior selection. Ants ascertain their conduct based on the exploration factor \(\alpha\), which varies from 0 to 1. A value of 0 signifies the complete absence of exploration, with ants exclusively opting for pathways with the highest pheromone concentration, whereas 1 signifies full-fledged exploration, opting for the least explored path. For values between 0 and 1, ants utilize the exploration factor as a probability metric for selecting different pathways within the path. To streamline the ants' pathway selection, a pathway width matrix is introduced. Each time a weight is selected by an ant, the corresponding pathway width increases. Consequently, when ants opt for exploration, they solely reference the corresponding pathway width to identify less explored paths. The probability formula for ant pathway selection is delineated as follows:  
\[\mathrm{P}_{\mathrm{k}}^{\mathrm{(i)}}= \begin{cases}\frac{1/C_{\mathrm{k}}^{ \mathrm{(i)}}}{\sum_{k=1}^{M}\tau_{\mathrm{k}}^{\mathrm{(i)}}/C_{\mathrm{k}}^{ \mathrm{(i)}}},&\mathrm{A}_{\mathrm{k}}^{\mathrm{(i)}}=1\\ \frac{\tau_{\mathrm{k}}^{\mathrm{(i)}}}{\sum_{k=1}^{M}\tau_{\mathrm{k}}^{ \mathrm{(i)}}},&\mathrm{A}_{\mathrm{k}}^{\mathrm{(i)}}=0\end{cases}  \tag{7}\]  
where \(\mathrm{P}_{\mathrm{k}}^{\mathrm{(i)}}\) signifies the likelihood of the ant selecting the i-th channel in the k-th stratum, \(\mathrm{C}_{\mathrm{k}}^{\mathrm{(i)}}\) represents the frequency of pathway exploration, and \(\tau_{\mathrm{k}}^{\mathrm{(i)}}\) signifies the pheromone concentration on that pathway. The path selection procedure by the ants encompasses the selection of channels or attention heads. In the pre-search stage, the number of path selections is small, fostering the expansion of pathway diversity. The conclusive phase is pathway assessment, where the paths chosen by each ant are utilized to execute soft pruning on the neural network. Subsequent to this, accuracy testing is applied to the validation set, and the attained test accuracy is adopted as the pheromone disseminated by the ant on the corresponding path. An apparent challenge in this procedure is the potential selection of the identical weight by multiple ants con currently, leading to scenarios where a weight, despite yielding lower scores in numerous evaluations, may exhibit a higher pheromone concentration compared to other weights due to multiple selections.