# Pruning Deep Neural Network Models via Minimax Concave Penalty Regression
## 5 Discussion  
With the widespread adoption of high-performance GPUs in the field of deep learning, both large and deep models have surpassed human limits across various domains. The extensive use of billions, or even trillions, of parameters along with computationally intensive operations confines these models to operation solely within laboratories and data centers. The fundamental requirements for real-time online devices handling deep neural networks include high precision and low latency. Consequently, compressing and accelerating models have become crucial areas of research in academia and industry. The focus remains on how to downsize these large models to facilitate deployment on mobile devices, attracting significant attention. He [28] was the pioneer in employing Lasso regression for model compression, removing redundant channels through this technique. In channel selection, Lasso applies a constant penalty strength to the regression coefficients. However, [PERSON] and [PERSON] [32] argued that optimizing penalized regression is simpler and more cost-effective than subset selection. Additionally, [PERSON] and [PERSON] [33] introduced sparsity to the original variables and incorporated sparse regression based on traditional principal component analysis. This not only retained the advantages of traditional principal component analysis but also enhanced model estimation accuracy due to its sparsity. [PERSON] et al. [34] proposed a variable selection method for the functional regression model in the case of sparse functional-type data. They performed functional principal component analysis on sparse functional-type independent variables based on conditional expectation. The estimated orthogonal eigenfunctions were then used as basis functions to expand the model. In contrast to [PERSON], [28] evidences the application of MCP regression for channel selection, deviating from the use of Lasso regression. This modification, introducing the MCP regularization function, validates the work's significance in this field. We also adhere to the use of maximum squares regression. The global pruning analysis, encompassing the single-layer pruning of the model, demonstrated the superior performance of MCP regression over Lasso regression. Comparative experiments were conducted on two datasets, CIFAR10 and CIFAR100, involving three networks: VGG19_Simple, VGG19, and ResNet34. Specifically, for VGG19_Simple trained on the CIFAR10 dataset, the Top1 accuracy of the MCP regression method after pruning improved by an average of 2.3% compared with that of the Lasso regression method. Due to the smaller dataset and increased model redundancy, the accuracy of the model remained higher after pruning. For the VGG19 model trained on the CIFAR100 dataset, where model redundancy is relatively low, channel selection becomes crucial. Here, the Top1 accuracy of the model after MCP regression increased by an average of 10% compared with that of the Lasso method, and the accuracy of Top1 after retraining improved by 0.67% at a pruning ratio of 0.3. For the ResNet34 model trained on the CIFAR100 dataset, with less model redundancy than VGG19, the Top1 accuracies of the MCP method improved by an average of 1.98% and 4.56% compared with that of the Lasso method after Lasso method pruning under two different strategies. In summary, all three ablation experiments consistently demonstrated the superiority of MCP regression for channel selection followed by output reconstruction over Lasso regression for channel selection followed by reconstruction. In the experimental section, it is evident that the MCP regression method yields a higher accuracy for the pruned model in comparison with MCP-L1. Additionally, the accuracy of the fine-tuned model approaches that of the original model. However, the computational time consumption and memory usage of the MCP regression method significantly surpass those of MCP-L1. Consequently, each method has its own set of advantages and disadvantages.