# Efficient Pruning of Detection Transformer in Remote Sensing Using Ant Colony Evolutionary Pruning
## 4 Experimental Results and Analysis
### Pruning Different Components  
Due to the complexity of the DETR-like neural network structure, we adopted different pruning strategies for the Backbone layer and Transformer layer. In this section, we verify these pruning strategies by pruning the Backbone and Transformer layers separately. All other experimental settings remain the same as before, and the experimental results are shown in Figure 6. Figure 6a is a data table, where \"_Transformer\" and \"_Backbone\" in the  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Models** & **Methods** & **Accuracy** & **Size (M)** & **MACs (G)** \\ \hline \multirow{4}{*}{DETR} & base & **81.20\%** & 41.6 & 56.77 \\  & L1-Norm & 77.50\% & 20.0 & 30.33 \\  & NS & 78.80\% & 19.6 & 29.19 \\  & ACEP (Ours) & 80.70\% & **19.1** & **28.95** \\ \hline \multirow{4}{*}{Deformable} & base & **90.60\%** & 40 & 114.81 \\  & L1-Norm & 87.90\% & 19.39 & 59.70 \\ \cline{1-1}  & NS & 88.10\% & 18.3 & 58.90 \\ \cline{1-1}  & ACEP (Ours) & 89.90\% & **18** & **57.4** \\ \hline \multirow{4}{*}{DINO} & base & **90.50\%** & 40.5 & 117.61 \\  & L1-Norm & 87.20\% & 20.7 & 63.45 \\ \cline{1-1}  & NS & 87.60\% & 20.5 & 62.87 \\ \cline{1-1}  & ACEP (Ours) & 89.70\% & **19.8** & **62.33** \\ \hline \hline \end{tabular} Note: The data in bold represents the optimal value for each item.  
\end{table}
Table 2: Comparison with other pruning methods.  
Figure 5: The experimental results. The green base points represent the data before pruning, and the blue pruned points represent the data after pruning.  
model names indicate pruning was only carried out on those layers. Figure 6b-d represent changes in accuracy, the number of parameters, and computational complexity before and after pruning. \"Base\" in the graph represents data related to the model before pruning, while \"Pruned\" represents the data after pruning.  
Compared to DETR, new models like Deformable DETR and DINO have reduced the dimensionality of the feed-forward layer from 2048 to 1024 and introduced more types of operations. From Figure 6a, it can be observed that although these operations reduce the model parameters, they also significantly increase the computational load.  
Figure 6b shows that when pruning only the Transformer layer in the three neural networks, the accuracy drops less compared to pruning only the Backbone layer. This implies that in DETR-like neural networks, the feature extraction capability of the Backbone layer has a greater impact on accuracy. Since the parameters of DETR-like neural networks are mainly concentrated in the Backbone layer, Figure 6c shows that pruning the Backbone can achieve a greater reduction in the number of parameters.  
On the other hand, looking at the changes in computational load in Figure 6d, since the Backbones of the three neural networks are the same, there is not much difference in computational load when pruning the Backbone layer. However, Deformable DETR and DINO accumulate computational load in the Transformer layer, so these two neural networks achieve more reduction in Multiply-Accumulate Operations (MACs) when pruning the Transformer layer.  
From the experiments in this section, it is evident that the different pruning strategies applied to the Backbone and Transformer layers have achieved excellent pruning results. They reduce the model size and computational load while maximizing accuracy retention. This also highlights that DETR-like neural networks have a lot of redundancy in the Transformer layer, providing guidance for future compression techniques for DETR-like neural networks.  
In addition, in terms of method generalization, the experiments in this section prove that our method has high potential for neural networks with either a CNN or Transformer structure. In the future, we also plan to conduct further validation on more types of network structures to further demonstrate the generalizability of the ACEP method.  
Figure 6: _Cont._