## 3 The Proposed Method
### Rank Selection Based on Sensitivity  
The purpose of rank selection is significant in CP decomposition. Insufficient compression would result from an excessively high rank, while too low a rank would lead to a significant decrease in accuracy that cannot be compensated for by fine-tuning. However, a definitive approach for determining the optimal tensor rank does not currently exist. Indeed, the computational problem of finding the rank is NP-hard [37]. Therefore, an essential idea is used for estimating the rank: the relationship between redundancy and compression rate is inversely proportional, so the relationship between redundancy and rank is directly proportional. Specifically, as the redundancy of a layer increases, the required rank of the layer decreases. To estimate the redundancy of each layer, a layer's sensitivity can be defined as its responsiveness to redundancy through compression. If a layer has a high level of sensitivity, it would be unfavorable to put the rank at a low value, and vice versa.  
We propose the use of a sensitivity-based rank-selection method. The sensitivity can be quantified by calculating the average of the Hessian trace. More specifically, the Hessian can be efficiently estimated using randomized numerical linear algebra approaches, such as [PERSON]'s approach [38]. Significantly, this method just needs the calculation of the Hessian on an input vector that is generated at random. The computational cost is equivalent to that of back-propagating the gradient. A similar method has been proposed by [8] in terms of quantization.  
Here, our focus is on the trace of a matrix \(H\in R^{d\times d}\). Next, with a random vector \(z\in R^{d}\) whose component is i.i.d. sampled Gaussian distribution, we can obtain:  
\[\text{Tr}(H)=\text{Tr}(HI)=\text{Tr}\big{(}H\mathbb{E}\big{[}z\mathbb{z}^{T} \big{]}\big{)}=E\big{[}\text{Tr}\big{(}H\mathbb{z}z^{T}\big{)}\big{]}=E\big{[} z^{T}Hz\big{]}, \tag{9}\]  
where \(I\) denotes the identity matrix. Then the Hutchinson algorithm [4] can be utilized to compute the Hessian trace:  
\[\text{Tr}(H)\approx\tfrac{1}{m}\sum\limits_{i=1}^{m}Z_{i}^{T}Hz_{i}=Tr_{Est}(H). \tag{10}\]  
Figure 1: Convolution layer and its CP decomposition. Each transparent box corresponds to the three-way tensor \(\mathcal{X}\), \(\mathcal{Z}\), \(\mathcal{Z}^{\prime}\), and \(\mathcal{Y}\) in Equations (6)–(8), with frontal sides corresponding to spatial dimensions. Arrows represent linear mappings and illustrate how scalar values on the right are computed. Small boxes correspond to single elements of the target tensor. (**a**) Original convolution layer. (**b**) CP decomposition of the convolution layer. Yellow tube, blue box, and red tube correspond to \(1\times 1\), \(\text{D}\times\text{D}\), and \(1\times 1\) convolutions in (6), (7), and (8), respectively.  
Finally, the rank of each layer is calculated in proportion to its sensitivity. By calculating and comparing the Hessian trace of each layer in the CNN, the ratio is able to measure the relative sensitivity and guide the selection of the rank for each layer. However, since we only know the ratio, the average rank of all layers must be selected arbitrarily.