## 4 Experimental Results
### Discussion
#### 4.4.2 Iterative Fine-Tuning Based on Sensitivity Order  
To investigate the effects of iterative fine-tuning based on sensitivity, we performed several ablation studies with ResNet-18 compression models on ImageNet. Other models and datasets can provide similar results.  
We start with one-shot fine-tuning, which means only fine-tuning once after all convolutional layers have been CP decomposed. We label this as CP-one-shot in Table 4. Meanwhile, to validate the performance of our approach, which uses sensitivity order to guide the process of iterative fine-tuning, we also conduct tests for iterative fine-tuning with random order, denoted as CP-random in Table 4.  
For a fair comparison, the other parameters for these different fine-tuning strategies remain the same, such as tensor rank in each individual layer and sensitivity-constrained optimization in CP decomposition. According to the results, the CP-one-shot strategy presents the lowest Top-1 and Top-5 accuracy compared with CP-random and our method in the same FLOPs reduction.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multirow{2}{*}{**Method**} & **Compression** & \multirow{2}{*}{**Top-5 Acc (\%)**} & **FLOPs** \\  & & **Method** & & **Reduction** \\ \hline \multirow{4}{*}{VGG-16} & AutoPruner [51] & Pruning & \(-1.49\) & \(3.79\times\) \\  & Standard Tucker [10] & Tucker & \(-0.50\) & \(4.93\times\) \\  & HT-2 [52] & Tucker & \(-0.65\) & \(5.26\times\) \\  & Ours & CP & \(-0.38\) & \(5.26\times\) \\ \hline \multirow{4}{*}{Resnet-18} & FPGM [53] & Pruning & \(-0.55\) & \(1.72\times\) \\  & DSA [54] & Pruning & \(-0.73\) & \(1.72\times\) \\  & DACP [20] & Pruning & \(-1.48\) & \(1.89\times\) \\  & Standard Tucker [10] & Tucker & \(-1.55\) & \(2.25\times\) \\  & MUSCO [49] & Tucker & \(-0.30\) & \(2.42\times\) \\  & TRP [50] & Low-Rank Matrix & \(-2.34\) & \(2.60\times\) \\  & Ours & CP & \(-0.16\) & \(2.61\times\) \\ \hline \multirow{4}{*}{Resnet-50} & TRP [50] & Low-Rank Matrix & \(-0.80\) & \(1.80\times\) \\  & Standard Tucker [10] & Tucker & \(-1.75\) & \(2.04\times\) \\  & AKECP [55] & Pruning & \(-2.30\) & \(2.62\times\) \\  & HRANK [18] & Pruning & \(-1.86\) & \(2.64\times\) \\  & HT-2 [52] & Tucker & \(-0.71\) & \(2.85\times\) \\  & AutoPruner [51] & Pruning & \(-1.62\) & \(2.94\times\) \\  & Ours & CP & \(-0.51\) & \(2.97\times\) \\ \hline \hline \end{tabular}
\end{table}
Table 3: Comparison with various compression methods for VGG-16, ResNet-18, and ResNet-50 on ImageNet dataset.  
On the contrary, our approach yields optimal accuracy for the ResNet-18 models. This indicates that sensitivity-based iterative fine-tuning combined with CP decomposition is the most effective strategy for restoring the loss of accuracy.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multirow{2}{*}{**Method**} & \multirow{2}{*}{**Top-1 Acc (\%)**} & \multirow{2}{*}{**Top-5 Acc (\%)**} & \multirow{2}{*}{**FLOPs**} \\  & & & & **Reduction** \\ \hline \multirow{3}{*}{Resnet-18} & CP-one-shot & \(-4.09\) & \(-2.57\) & \(2.61\times\) \\  & CP-random & \(-2.23\) & \(-1.31\) & \(2.61\times\) \\  & Ours & \(-1.28\) & \(-0.16\) & \(2.61\times\) \\ \hline \multirow{3}{*}{Resnet-50} & CP-one-shot & \(-6.27\) & \(-3.62\) & \(2.97\times\) \\  & CP-random & \(-4.07\) & \(-1.86\) & \(2.97\times\) \\ \cline{1-1}  & Ours & \(-1.34\) & \(-0.51\) & \(2.97\times\) \\ \hline \hline \end{tabular}
\end{table}
Table 4: Ablation results in iterative fine-tuning based on sensitivity order.  
Figure 4: Performance evaluation of ResNet-18 on ImageNet after only CP decomposing convolutional layer 1 of block 4, with and without sensitivity-constrained optimization with various ranks.  
Figure 3: Fine-tuning curves for ResNet-18 on ImageNet dataset after only CP decomposing convolutional layer 1 of block 4, with and without sensitivity-constrained optimization.