## 3 The Proposed Method
### Sensitivity-Constrained Optimization in CP Decomposition  
Tensor decomposition can be considered as an optimization problem, i.e., minimizing the difference between the decomposed tensor and the original tensor. In the optimization process of CP decomposition, there may be some problems. For instance, it is common to observe that two rank-one tensors with relatively high Frobenius norms cancel each other out when applying optimization algorithms to low-rank decomposition with nonunique CP decomposition [39]. This degeneracy is a common phenomenon in most CP decompositions of the convolutional layer. The presence of degeneracy often leads to instability problems when retraining a neural network with a CP format [12; 13]. In particular, it impairs the ability of a model to complete fine-tuning and preserve network stability. Therefore, a stable method is introduced from the work of [40] to perform sensitivity-based CP decomposition. To obtain a more stable decomposition, [40] proposed a correction to the decomposition optimization with less sensitivity. The study revealed that sensitivity might be used as a potential metric to evaluate the level of degeneracy in the CP decomposition. Consequently, we intend to quantify the sensitivity and perform sensitivity-constrained optimization in CP decomposition. When decomposing the tensor \(\mathcal{K}\) of convolutional kernels using CP decomposition, as previously mentioned, the kernel \(\mathcal{K}\) is considered an order-3 tensor \(\mathcal{K}=\left[\mathrm{A},\mathrm{B},\mathrm{C}\right]\) with dimensions \(T\times S\times D^{2}\). Due to the spatial dimensions of the tensor \(\mathcal{K}\) being so small, it is not necessary to decompose them. Subsequently, the sensitivity is defined as:  
\[S(\mathcal{K})=\lim_{\sigma^{2}\to 0}\tfrac{1}{R\sigma^{2}}E\big{\{}\parallel \mathcal{K}-\left[\left[\mathrm{A}+\delta\mathrm{A},\mathrm{B}+\delta\mathrm{ B},\mathrm{C}+\delta\mathrm{C}\right]\right]\parallel_{F}^{2}\big{\}} \tag{11}\]  
where \(\delta\mathrm{A}\), \(\delta\mathrm{B}\), and \(\delta\mathrm{C}\) denote elements with i.i.d. Gaussian distribution \(N\big{(}0,\;\sigma^{2}\big{)}\). The sensitivity could be quantified by calculating the expectation (\(E\{\cdot\}\)) of the normalized squared Frobenius norm of the difference. Essentially, the sensitivity of the tensor \(\mathcal{K}\) is a metric that is relative to perturbations in individual factor matrices. Additionally, CP decomposition with great sensitivity is often ineffective. Upon performing a simple calculation, the expectation (\(E\{\cdot\}\)) in Equation (11) may be represented in the following form:  
\[ \begin{array}{l}E\big{\{}\parallel\mathcal{K}-\left[\mathrm{A}+\delta \mathrm{A},\mathrm{B}+\delta\mathrm{B},\mathrm{C}+\delta\mathrm{C}\right] \parallel_{F}^{2}\big{\}}\\ =E\big{\{}\parallel\left[\delta\mathrm{A},\mathrm{B},\mathrm{C}\right]\parallel_ {F}^{2}\big{\}}+E\big{\{}\parallel\left[\mathrm{A},\delta\mathrm{B},\mathrm{C }\right]\parallel_{F}^{2}\big{\}}+E\big{\{}\parallel\left[\mathrm{A},\mathrm{ B},\delta\mathrm{C}\right]\parallel_{F}^{2}\big{\}}\\ =R\sigma^{2}\mathrm{tr}\Big{(}\Big{(}\mathrm{C}^{T}\mathrm{C}\Big{)}\otimes \left(\mathrm{B}^{T}\mathrm{B}\right)\Big{)}+R\sigma^{2}\mathrm{tr}\Big{(} \Big{(}\mathrm{C}^{T}\mathrm{C}\Big{)}\otimes\left(\mathrm{A}^{T}\mathrm{A} \right)\Big{)}\end{array} , \tag{12}\]  
where \(\otimes\) is the Hadamard element-wise product, while \"\(\mathrm{tr}\)\" refers to the trace of a matrix. Subsequently, we substitute this equation into the aforementioned Equation (11) to obtain the compact representation of sensitivity:  
\[