# Efficient Pruning of Detection Transformer in Remote Sensing Using Ant Colony Evolutionary Pruning
## 3 Methodology
### Population Initialization  
In this study, the search space refers to the DETR neural network, which exhibits a neural network structure depicted in Figure 2. The structure primarily comprises a Backbone layer that integrates Resnet50 and Transformer layers. The Backbone layer comprises 4 Layers, each encompassing a DS Block and multiple ST Blocks. Within the Transformer layer, there exist 6 Encoders and 6 Decoders, wherein each Encoder and Decoder integrates attention mechanisms and feed-forward layers. Broadly speaking, the selection of the architecture can be accomplished through binary sparse operators as described below:  
\[\text{a}=\sum_{\text{k}=1}^{\text{N}}\sum_{\text{i}=1}^{\text{M}_{\text{k}}} \text{S}_{\text{k}}^{(\text{i})}\text{o}_{\text{k}}\Big{(}\text{x}^{(\text{i} )}\Big{)}, \tag{2}\]  
where \(\text{S}_{\text{k}}^{(\text{i})}\in\{0,1\}\) denotes a binary sparse operator indicating the selection of architecture parameters \(\text{o}_{\text{k}}\Big{(}\text{x}^{(\text{i})}\Big{)}\); k and i represent the coordinates of the architecture parameters. We manage distinct modules based on variations in the neural network structure, a detailed elucidation of which will be provided subsequently. At this juncture, each defined architecture parameter within the search space corresponds to a distinct sparse operator \(\text{S}_{\text{k}}^{(\text{i})}\), where 1 signifies the selection of the architecture and 0 indicates discarding it. Notably, N denotes the index of the last layer in the neural network, and \(\text{M}_{\text{k}}\) denotes the maximum number of channels in the k-th layer. However, due to the varying output channels across different neural network layers, the quantities demonstrate diversity. Upon initialization, the initial population is segregated into 6 independent populations based on the parameter distribution, specifically located within each layer of the Backbone, Encoders, and Decoders of Transformer. To secure search efficiency and efficacy, the number of ants within each population is twice the total number of individuals in the population. Within the Backbone, the neural network is configured with multiple CNN blocks, signifying that Equation (2) can be reformulated as follows:  
\[\text{a}=\sum_{\text{k}=1}^{\text{N}}\sum_{\text{i}=1}^{\text{M}_{\text{k}}} \text{C}_{\text{k}}^{(\text{i})}\text{Conv}(\text{W}_{\text{k}},\text{O}_{ \text{k}-1}), \tag{3}\]  
Figure 2: Detection Transformer (DETR) neural network structure. where \(C_{k}^{(i)}\in\{0,1\}\) denotes the channel sparse operator, \(Conv(\cdot)\) signifies the convolution operation, and \(W_{k}\in\mathcal{R}^{1\times 1\times M_{k-1}\times M_{k}}\) represents the \(l\)-th layer convolutional kernel encompassing input channels \(M_{k-1}\) and output channels \(M_{k}\). The path selected by the ants corresponds to the output channels of each layer. To facilitate the path selection by ants, channel sparse operators are employed to log the path selection of each ant. Nevertheless, owing to the existence of residual connections, as depicted in Figure 3, the output channel numbers of the residual paths must align with those of Conv3. Abandoning the residual path would constrain the optimal upper limit of the pruning ratio. Consequently, we introduce twin constraints based on the channel sparse operator, referred to as twin-channel sparse operators, for pruning the residual paths. The twin constraint is explicated as follows:  
\[M_{Conv3}=M_{Res}=min(M_{Conv3},M_{Res}) \tag{4}\]  
where \(M_{Conv3}\) signifies the number of output channels in the Cov3 convolutional layer, and \(M_{Res}\) denotes the number of output channels in the residual path. Specifically, during path selection, the residual path and Conv3 are compelled to possess an equivalent number of channels. Equations (3) and (4) together are the twin-channel sparse operators.