# Pruning Deep Neural Network Models via Minimax Concave Penalty Regression
## 1 Introduction  
Deep learning stands as a pivotal research focus within the realm of machine learning, playing a crucial role in domains such as data mining and natural language processing. At the core of deep learning's evolution are deep neural networks, encompassing diverse network layers designed for the extraction of target feature information and the acceleration of computational processes [1]. Typically, the learning capacity of deep neural networks correlates positively with the depth of network layers, especially at higher computational volumes. However, this advantage comes with accompanying challenges, including elevated storage costs, increased computational demands, and heightened energy consumption. These factors impose limitations on the advancement of deep neural networks, particularly in the context of real-time processing tasks and deployment on mobile devices [2]. Primarily, deep neural networks encounter challenges related to storage space. Initial investigations revealed a positive correlation between the number of model parameters and the likelihood of convergence, leading to enhanced accuracy. For instance, the early LeNet-5 comprised approximately 430K parameters [3], while the subsequent VGG-16 model boasted a significantly larger parameter count of 138 M [4]. The substantial expansion of storage capacity in mobile devices poses a hindrance to the advancement of neural network technology. Additionally, in terms of computational aspects, deep neural networks present challenges of high computation complexity, coupled with excessive demand for storage space. For instance, during the neural network learning process, rapid convolutional operations on multi-dimensional matrices are imperative, involving a plethora ofmatrix operations [5]. Furthermore, in the domain of energy consumption, deep neural networks, characterized by multiple layers of interconnected neurons and variables, exhibit considerable storage and power demands across diverse embedded applications. Notably, the initial fully connected level of VGGNet encompasses 100 M, with a maximum mass of 140 M, contributing substantially to heightened energy consumption [6]. Confronted with the burgeoning storage requirements of deep neural networks, the training of extensive networks necessitates a substantial volume of computations and energy consumption. Moreover, given that the training of large networks is predominantly executed on high-performance servers or clusters, the direct application of these procedures to mobile devices becomes impractical. Consequently, research on techniques related to the compression and acceleration of deep neural networks has emerged to address these challenges. Recent studies, including references [7; 8; 9; 10], have also reviewed model compression methods in the last few years. The complexity of a deep neural network model is determined by its parameters, prompting researchers to prioritize a reduction in network parameters as a primary avenue for compressing and optimizing the model. [PERSON] and [PERSON] [11] proposed a classification of model compression methods, distinguishing between shallow compression and deep compression. Shallow compression encompasses filter-level pruning and knowledge distillation. Regarding the latter, [PERSON] and [PERSON] [12] contend that knowledge distillation methods treat the neural network holistically. These methods employ a learning approach to train a smaller network guided by a larger network with comparable performance, indirectly achieving the goal of compression. Due to the simplicity of the network architecture and the limited significance of the relationship between model performance and size in stochastic neural networks, [PERSON] and [PERSON] [13] proposed a self-distillation pipeline tailored for such networks.