# The Prediction Performance Analysis of the Lasso Model with Convex Non-Convex Sparse Regularization
## 4 Numerical Experiment
### Synthetic Data  
The data in this experiment are simulated with \(n=4000\) samples and \(p=8000\) features, where the correlation between features \(j\) and \(j^{\prime}\) is equal to \(0.6|j-j^{\prime}|\). The true vector \(\beta^{*}\) consists of 800 non-zero entries, all equal to 1. The observations are obtained through the linear model \(y=X\beta^{*}+\epsilon\), where \(\epsilon\) is Gaussian noise with variance such that \(\|X\beta\|/\|\epsilon\|=5\) (\(SNR=5\)). The parameter \(\lambda\) is expressed as a fraction of \(\lambda_{\text{max}}=\|X^{\top}y\|_{\infty}/n\). The objective function incorporates a maximum penalty \(\lambda_{\text{max}}\) for regularization, which will give the maximum sparse solution, i.e., \(\beta=0\). Then, the range of \(\lambda\) values can be set to vary between \((0,\lambda_{\text{max}})\) in order to obtain a regular path graph. For the GMC regularization, it is necessary to explicitly specify matrix \(B\) we set it by using (16) and \(\omega=0.8\). In this experiment, F1-score and root mean square error (RMSE) were chosen as evaluation metrics to assess the predictive performance of model (14). The F1 score is the harmonic mean of precision and recall, offers a more comprehensive perspective for assessing model performance in comparison to solely considering precision and recall metrics. Specifically, the precision and recall are defined as  
\[\text{Precision}=\frac{|\text{Supp}(\hat{\beta})\cap\text{Supp}(\beta^{*})|}{| \text{Supp}(\hat{\beta})|},\quad\text{Recall}=\frac{|\text{Supp}(\hat{\beta })\cap\text{Supp}(\beta^{*})|}{|\text{Supp}(\beta^{*})|}, \tag{31}\]  
where \(\hat{\beta}\) and \(\beta^{*}\) respectively represent the estimated value and the true value. Then, the F1 score is defined as  
\[\text{F1}=2\times\frac{\text{Precision}\times\text{Recall}}{\text{Precision}+ \text{Recall}}. \tag{32}\]  
The range of F1 score is between 0 and 1, where an F1 score of 1 indicates complete support for recovery, meaning the model accurately estimates sparse vectors; while an F1 score of 0 means no support for recovery, indicating zero estimation capability. Thus, a higher F1 score signifies stronger predictive performance demonstrated by the model. The RMSE is an important indicator in regression model evaluation, which can be used to measure the magnitude of prediction errors. Its definition is as follows  
\[\text{RMSE}=\sqrt{\frac{1}{n}\sum_{i=1}^{n}(\beta_{i}-\beta_{i}^{*})^{2}}, \tag{33}\]  
where \(\beta_{i}\) is the \(i\)-th predicted value, \(\beta_{i}^{*}\) is the \(i\)-th actual observed value, \(n\) is the number of observations. A smaller RMSE value indicates a higher predictive capability and better model fit. We compared the GMC sparse regularization (14) with the traditional \(\ell_{1}\) and \(\ell_{1/2}\) regularizations. To obtain typical conclusions, we conducted 100 Monte Carlo experiments using random noise with \(SNR=5\) and calculated the average F1 and RMSE values. As shown in Figure 1a, when an appropriate \(\lambda\) value is selected, the F1 score of the GMC sparse regularization reaches the maximum value of 1. In contrast, the F1 scores of the \(\ell_{1}\) and \(\ell_{1/2}\) regularizations are approximately 0.7 and 0.95, respectively, both lower than the performance of the GMC regularization. Additionally, it can be clearly observed from Figure 1b that in the sparse vector prediction task, the minimum RMSE of the GMC sparse regularization is significantly lower than that of the \(\ell_{1}\) and \(\ell_{1/2}\) regularizations. This result further validates the superior accuracy of the GMC sparse regularization in sparse vector prediction. On the other hand, Figure 1 shows that when estimating sparse vectors, the difference between the \(\lambda\) values corresponding to the maximum F1 score and the minimum RMSE in the Lasso model based on GMC sparse regularization is relatively small.