# The Prediction Performance Analysis of the Lasso Model with Convex Non-Convex Sparse Regularization
## 1 Introduction  
The Lasso model performs automatic feature selection by shrinking some coefficients to exactly zero. This mechanism prunes irrelevant features while retaining statistically significant predictors, thereby constructing a streamlined model with enhanced interpretability [1]. Consequently, the Lasso model has emerged as one of the most widely adopted tools for linear regression and finds applications across diverse domains including statistical and machine learning [2, 3, 4, 5, 6, 7], image and signal processing [8, 9, 10, 11]. Specifically, for \(n\) random observations \(y_{1},y_{2},...,y_{n}\in\mathbb{R}\) and \(p\) fixed covariates \(x_{1},...,x_{p}\in\mathbb{R}^{n}\). This high-dimensional linear model is defined as  
\[y=X\beta^{*}+\epsilon,\quad\epsilon\sim\sigma^{*}\mathcal{N}_{n}(0,I_{n}), \tag{1}\]  
where \(y:=(y_{1},y_{2}...,y_{n})^{\top}\in\mathbb{R}^{n}\) is the vector of observations, \(\beta^{*}\in\mathbb{R}^{p}\) is the unknown parameter of interest, \(X:=(x^{1},...,x^{p})\in\mathbb{R}^{n\times p}\) is a deterministic design matrix (for without loss of generality, we assume \(\|x\|_{2}^{2}\leq n\) for all \(j\in(1,...,p)\)), \(\epsilon\in\mathbb{R}^{n}\) is the noise vector with identically and independently distributed (i.i.d.) Gaussian entries with variance \(\sigma^{*2}\)and \(I_{n}\) denotes the identity matrix in \(\mathbb{R}^{n\times n}\). In general, we assume \(p>n\), and our goal is to accurately estimate \(\beta^{*}\in\mathbb{R}^{p}\). The Lasso model is a type of linear regression model utilized for estimating \(\beta\) and can be formulated as the following optimization problem [12]  
\[\hat{\beta}_{\lambda}^{L1}\in\arg\min_{\beta}\frac{1}{2n}\|y-X\beta\|_{2}^{2}+ \lambda\|\beta\|_{1}, \tag{2}\]  
where \(\frac{1}{2n}\|y-X\beta\|_{2}^{2}\) is the mean squared error (MSE) loss and the \(\ell_{1}\) norm \(\|\beta\|_{1}\) is the regularizer, \(\lambda>0\) is a predefined tuning parameter that controls the level of regularization. The prediction performance of the Lasso model refers to how well it predicts outcomes or responses based on input features and depends on its ability to strike a balance between regularization and feature selection. The prediction performance of the Lasso model plays a crucial role in understanding its effectiveness [12; 13; 14]. The \(\ell_{1}\) norm in Equation (2) is the most popular regularization because of its outstanding ability to induce sparsity among convex regularization methods. However, it has been observed that this formal approach often underestimates the high-amplitude components of \(\beta\)[15]. In contrast, non-convex regularizations in the Lasso model have also made significant progress [6; 10; 16; 17]. For instance, the smoothly clipped absolute deviation (SCAD) penalty [18] and the minimax concave penalty (MCP) [15], as well as \(\ell_{q}(0<q<1)\) and other non-convex regularization terms, can more accurately estimate high-amplitude components. However, due to their non-convex nature, the objective function is prone to getting stuck in local optima, which poses additional challenges to the solution process. To utilize the advantages of both non-convex regularization and convex optimization methods, [PERSON] et al. introduced the CNC strategy, which involves constructing a non-convex regularizer by subtracting the smooth variation from its convex sparse counterpart [19; 20; 21; 22]. Under specific conditions, the proposed regularization ensures global convexity of the objective function. Due to its global convexity, CNC sparse regularization can effectively avoid local optima and overcome the biased estimation associated with nonconvex sparse regularization. As a result, it has gained widespread usage in image processing and machine learning applications [23; 24; 25; 26; 27; 28]. However, to the best of the authors' knowledge, most of the research on CNC sparse regularization focuses on algorithm design and applications and the theoretical analysis is lacking.