# Efficient Pruning of Detection Transformer in Remote Sensing Using Ant Colony Evolutionary Pruning
## 3 Methodology
### Global Search and Bayesian Pruning  
During the preliminary exploration phase, various ant colonies establish optimal local architectures within the search space; however, the ultimate objective is to identify the globally optimal sparse sub-model. Consequently, transitioning from local to global exploration becomes imperative. Initially, the pheromone matrices of the identified sub-modules from the earlier exploration phase are consolidated into a unified global pheromone matrix. Subsequently, the colonies of ants are initialized. Differing from the preliminary exploration phase, a minor fraction of ants within each ant colony is metamorphosed into elite ants to augment competitiveness owing to inter-colony rivalry. In order to enhance the diversity of exploration paths, if a specific path is chosen by elite ants, ordinary ants will refrain from entering that path and opt for an alternate path at random. Nevertheless, elite ants originating from other colonies can still select that path given their equivalent competitive status. The exploration factor of ants will progressively intensify in line with the spatial separation between modules, transitioning from proximal to distal, while the elite ants will consistently uphold a diminished level of exploration. Throughout global exploration, in consideration of the attributes of remote sensing objectives, we introduce pruning ratios into the weights of each layer as environmental restrictions. This implies that ants will determine the number of weights based on the environmental constraint during weight selection. Intending to automate the exploration process, Bayesian optimization is implemented to autonomously define the sparsity of each layer within the neural network. Bayesian optimization is a universal optimization approach particularly suited for intricate functions with substantial computational expenses. Once the Bayesian optimizer assigns an appropriate pruning ratio for each layer, the ants will select the weights for each layer corresponding to this ratio. The assessment metric of Bayesian optimization encompasses an efficiency coefficient articulated as follows:  
\[\eta=\frac{1-\sigma}{(\text{Para}-\text{Para}^{*})}, \tag{8}\]  
where \(\sigma\) signifies the test accuracy of the pruned model, while Para and Para\({}^{*}\) denote the count of model parameters pre- and post-pruning, respectively. The target parameter of optimization is the vector X signifying the pruning ratios for each layer of the neural network. In order to prevent extreme pruning ratios, the scope of the pruning ratios during the optimization process is confined to [0.1, 0.9], and initial sampling points are randomly generated within this range. Additionally, a Gaussian process is employed as a surrogate model during the optimization process. Gaussian processes are adept at managing uncertainty and offering confidence intervals for the objective function, ensuring efficacious global optimization. The Gaussian process is denoted by the following:  
\[\eta\sim\text{GP}\big{(}\text{m}(\text{x}),\text{k}\big{(}\text{X},\text{X}^{ \prime}\big{)}\big{)}, \tag{9}\]  
where m(x) embodies the anticipated value at sampling point X, and k\(\big{(}\text{X},\text{X}^{\prime}\big{)}\) represents the covariance between point X and X\({}^{\prime}\), wherein the covariance is ascertained by the following Gaussian kernel function:  
\[\text{k}\big{(}\text{X},\text{X}^{\prime}\big{)}=\delta^{2}\text{e}^{-\frac{ \big{(}\text{X}-\text{X}^{\prime}\big{)}^{2}}{2\text{X}}}, \tag{10}\]where the amplitude \(\delta\) controls the overall fluctuation amplitude and the length metric \(\lambda\) controls the overall smoothness. The formulation of the Gaussian process illustrates that the combined distribution of the observation vector \(\eta\) and the projected value \(\eta^{*}\) also adheres to a Gaussian distribution, indicating that the efficiency coefficient of any point can be predicted without actual pruning.