## 4 Ablation Study
### Effects of Threshold Selection  
In this subsection, we explore the influence of different threshold values on the performance of the pruned model. We tested TeranusNet on the Inria dataset with two different threshold values, and the corresponding experimental results are reported in Table 8. Here, importance weights applied to each filter were considered to define a threshold value. To study the influence of different thresholds, we set \(\varepsilon=0.4\), as a larger value led to an unstable training process in the case of using the mean of the importance weights as a pruning threshold. If weak filters were removed by applying a global threshold, then the pruned model lost its ability to accurately classify by removing an entire layer. Therefore, pruning thresholds were adapted locally for each layer. The pruning thresholds used represent either the mean importance weight of the filters per layer or 1.2-times the average importance weight of the filters per layer. As shown in Table 8, the mean of the importance weights taken as the threshold provided a good performance under different initialization strategies, but failed to significantly reduce the FLOPs and the number of parameters. In contrast, defining the pruning threshold as 1.2-times the average importance weight, we were able to discard large amounts of filters while maintaining the accuracy intact. For instance, the mean of the importance weights used as the pruning threshold reduced 74.78% of the FLOPs under random initialization, whereas 1.2-times the average importance weight used as the pruning threshold reduced 84.34% of the FLOPs under the same  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Initialization Strategy** & \(\varepsilon\) & **Val. Acc. (\%)** & **FLOPs \(\downarrow\) (\%)** & **\# Param. \(\downarrow\) (\%)** \\ \hline \multirow{8}{*}{Random} & 0.3 & 95.84 & 84.34 & 83.50 \\ \cline{2-5}  & 0.35 & 95.85 & 84.34 & 83.50 \\ \cline{2-5}  & 0.4 & 95.77 & 84.34 & 83.50 \\ \cline{2-5}  & 0.45 & 95.78 & 84.34 & 83.50 \\ \cline{2-5}  & 0.5 & 95.68 & 84.98 & 83.54 \\ \hline \multirow{8}{*}{Chaotic (logistic map)} & 0.3 & 95.92 & 80.59 & 81.12 \\ \cline{2-5}  & 0.35 & 95.82 & 80.59 & 81.12 \\ \cline{2-5}  & 0.4 & 95.83 & 80.59 & 81.12 \\ \cline{2-5}  & 0.45 & 95.81 & 80.59 & 81.12 \\ \cline{2-5}  & 0.5 & 95.91 & 80.59 & 81.12 \\ \hline \multirow{8}{*}{He} & 0.3 & 95.99 & 75.84 & 75.42 \\ \cline{2-5}  & 0.35 & 95.91 & 75.84 & 75.42 \\ \cline{2-5}  & 0.4 & 95.80 & 77.30 & 75.50 \\ \cline{2-5}  & 0.45 & 96.03 & 64.18 & 73.20 \\ \cline{2-5}  & 0.5 & 96.04 & 64.18 & 73.20 \\ \hline \multirow{8}{*}{LeCun} & 0.3 & 95.84 & 74.96 & 74.27 \\ \cline{2-5}  & 0.35 & 95.97 & 74.96 & 74.27 \\ \cline{2-5}  & 0.4 & 95.93 & 74.96 & 74.27 \\ \cline{2-5}  & 0.45 & 95.75 & 76.48 & 74.66 \\ \cline{2-5}  & 0.5 & 96.00 & 66.02 & 72.25 \\ \hline \multirow{8}{*}{[PERSON]} & 0.3 & 95.88 & 75.84 & 75.42 \\ \cline{2-5}  & 0.35 & 96.03 & 75.84 & 75.42 \\ \cline{1-1} \cline{2-5}  & 0.4 & 95.82 & 77.30 & 75.50 \\ \cline{1-1} \cline{2-5}  & 0.45 & 96.07 & 64.18 & 73.20 \\ \cline{1-1} \cline{2-5}  & 0.5 & 96.04 & 64.18 & 73.20 \\ \hline \hline \end{tabular}
\end{table}
Table 7: Ablation study over the \(\varepsilon\) values for TeranusNet on Inria dataset.  
initialization strategy, which is an almost 10% larger reduction of the FLOPs. Hence, based on the experimental results, we adopted 1.2-times the mean of the importance weights as a proper pruning threshold, which provided acceptable performance while significantly reducing the FLOPs.