###### Abstract
For this reason, penalized methods have been recommended for prediction [26; 27]. However, researchers have argued that they should not be viewed as a solution to small sample sizes or low SNR. Instead, they should be used when a sufficiently large training dataset is available and the SNR is moderate. These conditions help to reduce uncertainty in the estimation of tuning parameters, which control the quality of regression estimates and, consequently, the prediction accuracy of the models [7; 28]. Model selection criteria such as cross-validation (CV), Akaike information criterion (AIC) [29], and Bayesian information criterion (BIC) [30] have been proposed for selecting tuning parameters in penalized methods [31; 32], as well as for choosing the best models in classical methods [33; 34; 35]. In this study, we employed these three popular criteria, which target different types of models. CV and AIC aim to select the best model for prediction on new unseen data, while BIC aims to identify the true data-generating model or the model that is closest to it [35]. Some studies have shown that BIC may outperform AIC in prediction when there are a few large effects and all other covariates are noise. In such situations, BIC applies a heavier penalty for model complexity, favoring simpler models that retain only covariates with large effects, while AIC may retain many noise variables [36]. Conversely, AIC may perform better than BIC when there are a few covariates with large effects, followed by many covariates with smaller effects that gradually decrease. In such settings, AIC's penalty for model complexity is less severe than BIC's, allowing it to more effectively capture these smaller effects and achieve better predictive performance [35; 36]. Since CV is a widely used approach for selecting tuning parameters, our third objective (O3) is to compare the prediction accuracy and model complexity of classical and penalized regression methods tuned using this approach. Our fourth objective (O4) is to assess how the three model selection criteria (CV, AIC, and BIC) influence both prediction accuracy and model complexity of variable selection methods. The performance of variable selection methods is known to be sensitive to the proportion of noise variables [9]. Therefore, our fifth objective (O5) is to evaluate the robustness of the approaches in settings with a high proportion of noise variables. Throughout this paper, we standardized each covariate in the training data to have a mean of zero and unit variance. Additionally, we centered the response variable by its mean to omit the intercept from the model. All variables in the new dataset were standardized using the statistics derived from the training data. The rest of the paper is organized as follows: Section 2 provides a summary of the simulation design, following the ADEMP structure (Table 1), which entails defining aims (A), data-generating mechanisms (D), estimands/targets of analysis (E), methods (M), and performance (P) measures [37]. In addition, it describes the methods used, including their tuning parameters and initial estimates for two-stage approaches (NNG and ALASSO). Section 3 presents the results from the simulation studies, including a detailed summary in a structured format (Table 2), organized according to the five objectives outlined in the \"Aims\" section of the ADEMP. Section 4 presents the results of a real data example. Section 5 contains the discussion, Section 6 provides the conclusions, and Section 7 outlines some directions for future research.