## 2 Materials and Methods
### Hyperparameter Tuning and Model Evaluation  
#### 2.3.1 Hyperparameter Optimization Approaches  
To systematically select hyperparameters for each model, stratified \(k\)-fold cross-validation was employed. Stratification preserves the class distribution in every fold, mitigating the risk of model bias toward a particular class distribution, which is especially crucial in binary classification problems [28]. Moreover, \(k\)-fold cross-validation offers a reliable estimate of model performance by rotating the training and validation sets through  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Model** & **Hyperparameter (Abbreviation)** \\ \hline Logistic Regression (LR) & Solver (S), Penalty (P), C, L1 Ratio (LR), Max Iterations (MI) \\ \hline Random Forest (RF) & Estimators (N), Max Depth (MD), Min Samples Split (MSS), Min Samples \\  & Leaf (MSL), Max Features (MF), OOB Score (OOB), Max Samples (MS) \\ \hline XGBoost (XGB) & Estimators (N), Max Depth (MD), Learning Rate (LR), Subsample (SS), \\  & Colsample ByTree (CT), Colsample ByLevel (CL), Gamma (G), Min Child \\  & Weight (MCW), Max Delta Step (MDS), Reg Alpha (RA), Reg Lambda (RL) \\ \hline LightGBM (LGBM) & Estimators (N), Max Depth (MD), Learning Rate (LR), Subsample (SS), \\  & Colsample ByTree (CT), Min Data in Leaf (MDL), Min Split Gain (MSG), \\  & Feature Fraction (FF), Reg Alpha (RA), Reg Lambda (RL), Verbose (V) \\ \hline CatBoost (CB) & Iterations (I), Depth (D), Learning Rate (LR), Subsample (SS), RSM, Min Data \\  & in Leaf (MDL), L2 Leaf Reg (L2R), Bagging Temp (BT), Random Strength (RS), \\  & Grow Policy (GP), Leaf Estimation Iteration (LEI), Eval Metric (EM) \\ \hline Support Vector Machines (SVM) & Kernel (K), C, Gamma (G), Degree (D), Tolerance (T), Max Iterations (MI) \\ \hline Artificial Neural Networks (ANN) & Solver (S), Hidden Layer Sizes (HLS), Activation (AT), Alpha (A), Learning \\  & Rate (LR), Max Iterations (MI), Batch Size (BS), Beta 1 (B1), Beta 2 (B2), \\  & Epsilon (E), Tolerance (T), Early Stopping (ES), Validation Fraction (VF) \\ \hline \hline \end{tabular}
\end{table}
Table 1: Selected hyperparameters for ML models.  
each fold. A consistent random seed was used to ensure the reproducibility of these splits. Cross-validation was chosen in lieu of a simple train-validation-test partition because it maximizes data usage for both training and validation, reducing variance in estimates of model performance.  
Multiple hyperparameter optimization techniques were explored--manual tuning, grid search, random search, nested cross-validation, and Bayesian optimization. Nested cross-validation provides an unbiased performance estimate by nesting an inner optimization loop within an outer cross-validation loop [29]. However, this approach proved excessively time-consuming given the extensive search space across numerous models and parameters. Consequently, Bayesian Optimization (implemented via Hyperopt, without nesting) emerged as the primary optimization strategy, balancing search efficiency with overall computational cost [30].  
Hyperopt utilizes probabilistic models to explore the hyperparameter space, typically yielding superior performance in fewer iterations than exhaustive methods [31]. This method employs the Tree-structured Parzen Estimator (TPE) to adaptively sample promising regions of the parameter space, often discovering near-optimal configurations without the prohibitive time investment required by more exhaustive search methodologies. A custom search space was defined for each model, encompassing both integer and continuous parameters while accounting for conditional dependencies (i.e., certain hyperparameters became active only when specific model settings were chosen).