## 2 Related Works
### Compression of Convolutional Neural Networks  
Various methods have been developed to make convolutional neural networks (CNNs) smaller, faster, and more energy-efficient. Below, we briefly review key CNN compression approaches. Pruning is one of the most commonly used techniques for the compression of CNNs. It works by setting specific weights to zero and can be categorized as either unstructured or structured pruning. Unstructured pruning achieves high compression ratios and accuracy [25], but the resulting irregular sparsity patterns limit its acceleration on hardware. Structured pruning, such as filter and channel pruning [26; 27; 28], is more hardware-friendly but often sacrifices compression ratio and accuracy due to enforced structural patterns. Sparsification is another popular approach to CNN compression. Sparsity can be applied at multiple levels, such as weights, filters, or channels [29; 30; 31]. Unstructured sparsity achieves better compression and accuracy [32], but it presents challenges for hardware implementation, including irregular memory access. Structured sparsity, while more suitable for hardware, generally results in lower compression ratios and reduced accuracy compared to unstructured approaches [33]. Quantization is widely used in CNN compression, reducing the size of the model by representing weights and activations with a limited number of bits, with precision depending on resource allocation and accuracy requirements [34]. Tensor decomposition compresses CNNs by factorizing weight tensors into compact representations. Tensor decomposition methods, such as CP, Tucker, and SVD, provide a deterministic approach to compressing CNNs by approximating the weight tensors of neural network layers using low-rank tensor representations. The primary advantage of tensor decomposition is that it replaces a layer with a compressed representation that closely approximates the original weight tensor, with the approximation error minimized through discrete optimization techniques. Compared to quantization and pruning, tensor decomposition methods achieve higher compression ratios by effectively reducing the dimensionality of weight tensors while preserving the layer's functionality. This makes tensor decomposition particularly advantageous for achieving compact and efficient models. Notable contributions in this field include [35], which demonstrated the use of Tensor Train Decomposition to compress Fully Connected Layers in CNNs, and [36], which extended the application of Tensor Train Decomposition to Convolutional Layers in Deep Neural Networks. In addition, various methods such as decompositions of the tensor ring (TR) [37] and Tucker [11] have been employed to significantly reduce the number of model parameters. Recent advances include a low-rank Tucker-CP decomposition [38], which improves weight tensor approximation in convolutional layers, enabling effective compression while maintaining high accuracy. However, high compression ratios of tensor decomposition methods also come with a downside: they require fine-tuning the model after compression to minimize the degradation of performance on downstream tasks. For example, [38] presents a hybrid approach combining single-layer fine-tuning and full-network fine-tuning after applying compression. The novelty and strength of our approach lie in combining multiple decomposition methods rather than applying a single decomposition technique to all layers of the network. Using a combination of CP, Tucker, and SVD decompositions allows us to better tailor the compression process to each layer, as different layers of a neural network may be more effectively approximated by specific decomposition methods. This hybrid approach reduces the overall performance degradation typically associated with compression. In addition, we enhance the framework for compression model fine-tuning to further mitigate performance loss.