## 3 The Proposed Method  
### CP Decomposition for Convolutional Layers  
In the process of CP decomposition, a tensor is decomposed by CP into a linear combination of rank-one tensors. We focus on the three-way tensor and consider that a tensor \(\mathcal{T}\) is denoted as:  
\[\mathcal{T}=[\mathrm{A},\mathrm{B},\mathrm{C}], \tag{1}\]  
where A, B, and C denote factor matrices of tensor \(\mathcal{T}\). The notation \([\dots]\) denotes that:  
\[\mathcal{T}=\sum\limits_{r=1}^{R}\mathrm{a}_{r}\otimes\mathrm{b}_{r}\otimes \mathrm{c}_{r}, \tag{2}\]  
where \(\otimes\) is the outer product of the tensors. The decomposition of the tensor \(\mathcal{T}\) into its \(R\) components is shown in Equation (2). The tensor rank is the overall amount of components in the decomposed tensor, denoted by \(R\). The rank \(R\) is a determining factor in the extent of weight reduction in a convolution layer. Specifically, a lower value of \(R\) corresponds to a greater weight decrease. \(\{\mathrm{a}_{r}\}\), \(\{\mathrm{b}_{r}\}\), and \(\{\mathrm{c}_{r}\}\) denote the columns of factor matrices A, B, and C, respectively.  
Then, we focus on the tensor of the convolution layer. Convolution layers in CNNs typically transform an input tensor \(\mathcal{X}\), with dimensions \(S\times W\times H\), into an output tensor \(\mathcal{Y}\), with dimensions \(T\times W^{\prime}\times H^{\prime}\). This transformation is achieved by utilizing a four-dimensional kernel tensor \(\mathcal{K}\), with dimensions \(T\times S\times D\times D\). We assume that kernel tensors are square-shaped and have an odd dimension \(D\):  
\[\mathcal{Y}_{t,w^{\prime},h^{\prime}}=\sum\limits_{s=1}^{S}\sum\limits_{j=1}^{ D}\sum\limits_{i=1}^{D}\mathcal{K}_{t,s,j,i}\mathcal{X}_{s,w_{j},h_{i}}. \tag{3}\]  
Now, the current problem involves the approximation of the tensor \(\mathcal{K}\) using CP decomposition with rank \(R\). This is illustrated by Equation (4). The spatial dimensions of tensor \(\mathcal{K}\) are not decomposed due to their relatively modest size, such as \(3\times 3\) or \(5\times 5\):  
\[\mathcal{K}_{t,s,j,i}=\sum\limits_{r=1}^{R}\mathrm{U}_{r,s}^{(1)}\mathcal{U}_ {r,j,i}^{(2)}\mathrm{U}_{t,r}^{(3)}, \tag{4}\]  
where \(\mathrm{U}_{r,s}^{(1)}\), \(\mathcal{U}_{r,j,i}^{(2)}\), and \(\mathrm{U}_{t,r}^{(3)}\) are the three tensors with sizes \(R\times S\), \(R\times D\times D\), and \(T\times R\), respectively.  
By substituting Equation (4) into Equation (3), we obtain an approximate method to calculate the convolution (3):  
\[\mathcal{Y}_{t,w^{\prime},h^{\prime}}=\sum\limits_{r=1}^{R}\mathrm{U}_{t,r}^{ (3)}\left(\sum\limits_{i=1}^{D}\sum\limits_{i=1}^{D}\mathcal{U}_{r,j,i}^{(2)} \left(\sum\limits_{s=1}^{S}\mathrm{U}_{r,s}^{(1)}\mathcal{X}_{s,w_{j}h_{i}} \right)\right). \tag{5}\]  
As illustrated in Figure 1, Equation (5) denotes that the output tensor \(\mathcal{Y}\) is derived by a series of distinct and compact convolutions applied to the input tensor \(X\):  
\[\mathcal{Z}_{r,w,h}=\sum\limits_{s=1}^{S}\mathrm{U}_{r,s}^{(1)}\mathcal{X}_{ s,w,h}, \tag{6}\]  
\[\mathcal{Z}_{r,w^{\prime},h^{\prime}}^{\prime}=\sum\limits_{j=1}^{D}\sum \limits_{i=1}^{D}\mathcal{U}_{r,j,i}^{(2)}\mathcal{Z}_{t,w_{j}h_{i}}, \tag{7}\]  
\[\mathcal{Y}_{t,w^{\prime},h^{\prime}}=\sum\limits_{r=1}^{R}\mathrm{U}_{t,r}^{ (3)}\mathcal{Z}_{r,w^{\prime},h^{\prime}}^{\prime}, \tag{8}\]  
where \(\mathcal{Z}_{r,w,h}\) and \(\mathcal{Z}_{r,w^{\prime},h^{\prime}}^{\prime}\) are intermediate tensors with dimensions \(R\times W\times H\) and \(R\times W^{\prime}\times H^{\prime}\), respectively.