## 2 Materials and Method
### Variable Selection Methods
#### 2.4.2 Penalized Regression Methods  
Nonnegative Garrote  
The NNG is a two-stage approach proposed for low-dimensional data. In the first stage, the initial estimates, \(\hat{\beta}^{init}\), are estimated using least squares or standard maximum likelihood estimation methods. In the second stage, the nonnegative shrinkage factors, \(\hat{\epsilon}(\lambda)\), are estimated. After obtaining the shrinkage factors, the NNG regression estimate for covariate \(x_{j}\) is calculated as \(\hat{\beta}_{j}^{NNNG}(\lambda)=\hat{\epsilon}_{j}(\lambda)\hat{\beta}_{j}^{init}\), which depends on the initial estimates and the tuning parameter \(\lambda\)[12; 17; 19]. The original NNG depends on OLS estimates as initial estimates. However, [PERSON] and [PERSON] [17] argue that NNG is a general procedure that can be used with other initial estimates such as ridge, lasso, and elastic net [48]. Ridge initial estimates are preferred over OLS estimates when covariates are highly correlated because they are more stable [15]. For a linear regression with standardized covariates and a centered response variable, the shrinkage factors, \(\hat{c}(\lambda)=\big{(}\hat{\epsilon}_{1}(\lambda),\ldots,\ \hat{c}_{p}(\lambda)\big{)}^{T}\), are obtained by minimizing the following objective function:  
\[\frac{1}{2n}\sum_{i=1}^{n}\left(y_{i}-\sum_{j=1}^{p}c_{j}\,x^{*}_{ij}\right)^{ 2}+\lambda\sum_{j=1}^{p}c_{j},\ \ c_{j}\geq 0,\ \ \lambda\geq 0,\]  
Lasso  
The lasso is a popular penalized regression approach, due to its prediction accuracy and computational efficiency [16; 49; 51]. It is a general procedure that can be applied to a broad variety of models, including generalized linear models [52; 53] and Cox models for censored survival data [54]. For a linear regression with standardized covariates and a centered response variable, the lasso estimates are obtained by minimizing the following objective function:  
\[\frac{1}{2n}\sum_{i=1}^{n}\left(y_{i}-\sum_{j=1}^{p}\beta\kappa_{ij}\right)^{ 2}+\lambda\sum_{j=1}^{p}|\beta_{j}|,\lambda\geq 0.\]  
The \(\updownarrow_{1}\) penalty term continuously shrinks the regression coefficients toward zero as the tuning parameter, \(\lambda\), increases. When \(\lambda\) is sufficiently large, some coefficients are forced to be exactly zero, thus conducting variable selection [11; 15]. However, the lasso penalty term shrinks small and large coefficients equally, which can lead to large bias, particularly for large coefficients, resulting in suboptimal predictions [15]. The ALASSO and RLASSO were proposed to reduce this bias and improve model selection accuracy. The lasso was tuned over 100 values of \(\lambda\), and one-dimensional 10-fold CV, AIC, and BIC were used to select the tuning parameters. Adaptive Lasso  
The ALASSO is a two-stage approach proposed to enhance the performance of the lasso, particularly in variable selection [15]. It replaces the \(\updownarrow_{1}\) penalty term of the lasso with a re-weighted version [15]. Similar to NNG, the data-dependent weights are calculated using the initial estimates. These weights impose a heavy penalty on coefficients with small initial estimates and a smaller penalty on coefficients with large initial estimates [15; 49]. By utilizing these weights, the ALASSO can reduce the bias of large effects and improve variable selection accuracy [15; 16]. For a linear regression with standardized covariates and a centered response variable, the ALASSO estimates are obtained by minimizing the following objective function:  
\[\frac{1}{2n}\sum_{i=1}^{n}\left(y_{i}-\sum_{j=1}^{p}\hat{\beta}_{j}\;x_{ij} \right)^{2}+\lambda\sum_{j=1}^{p}w_{j}\big{|}\beta_{j}\big{|},\;\lambda\geq 0,\]  
where the weights \(\;w_{j}=\big{|}\hat{\beta}_{j}^{init}\big{|}^{-\gamma}\) are a function of initial estimates \(\hat{\beta}_{j}^{init}\) and \(\gamma>0\). For comparison, we employed the same three initial estimates used in NNG for ALASSO.