# Efficient Pruning of Detection Transformer in Remote Sensing Using Ant Colony Evolutionary Pruning
## 4 Experimental Results and Analysis
### Experimental Setup and Results  
In this section, we use three neural networks, DETR, DINO, and Deformable DETR, to verify the effectiveness of the method. Each model used the intermediate feature extraction layer of Resnet50 as the Backbone and had six Encoders and six Decoders in the subsequent Transformer layers. Due to the different processing methods in the Transformer layer for the three neural networks, there were differences in the initial parameter quantity and MACs. Before pruning, the three models were pretrained using the HRSC2016 dataset, and then pruning was performed based on the pretrained models. During training, all three neural networks used the Adam optimizer; we set the maximum training epoch to 200 and the initial learning rate to \(1\times 10^{-4}\), which was reduced by 0.1 at 80 and 160 epochs. In the fine-tuning stage, we again used the Adam optimizer with a lower learning rate of \(1\times 10^{-6}\), training for 30 epochs. Furthermore, the batch size for all experiments during training and testing was set to two.  
When performing ACEP, to balance algorithm performance and computational resources, we set the pruning rate to 0.1 and the iteration times to 15 during the pre-search phase. In the global search phase, the number of elite ants in each population was set to four, and four ants died in each iteration. During Bayesian pruning, to balance smooth fitting and flexible responsiveness, the Gaussian kernel function's amplitude \(\delta\) was set to 0.5, and the length metric \(\lambda\) was set to 1. To allow the model to have a larger exploration space in the initial stage, the fluctuation coefficient of the Gaussian kernel function \(\beta\) was set to two.  
After the search, the experimental results are shown in Table 1 and Figure 5. From the results, it is observed that the pruned models have significantly fewer parameters and computational resources than the initial models, leading to a significant improvement in neural network inference efficiency when using pruned models in practical applications. Among the three neural networks, Deformable DETR achieved the best compression effect, with a 55% reduction in parameters and a 57.41 G reduction in computational resources, while the accuracy decreased by only 0.7%. DETR and DINO also achieved excellent pruning effects, with DETR reducing parameters by 54% and computational resources by 27.82 G. The DINO model reduced parameters by 51% and computational resources by 55.28 G, with a slight accuracy drop of 0.8%. The experiments proved that this method has excellent compression effects on DETR-like neural networks.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \multirow{2}{*}{**Models**} & \multicolumn{2}{c}{**Accuracy**} & \multicolumn{2}{c}{**Size (M)**} & \multicolumn{2}{c}{**MACs (G)**} \\ \cline{2-7}  & **Base** & **Pruned** & **Base** & **Pruned** & **Base** & **Pruned** \\ \hline DETR & 81.20\% & 80.70\% & 41.6 & 19.1 & 56.77 & 28.95 \\ Deformable DETR & 90.60\% & 89.90\% & 40 & 18 & 114.81 & 57.4 \\ DINO & 90.50\% & 89.70\% & 40.5 & 21.1 & 117.61 & 62.33 \\ \hline \end{tabular}
\end{table}
Table 1: Experimental results.