## 3 Methods and Materials
### Models Development and Training
#### 3.5.2 Hyperparameter Optimization
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Model Name** & **Hyperparameter Name** & **Definition** & **Hyperparameter Value Range** \\ \hline \multirow{4}{*}{Support Vector Machine (SVM)} & C value & The penalty parameter & (0.1, 0.8, 0.9, 1, 1, 1, 1, 2, 1, 3, 1.4, 0.25, 0.5, 0.75, 10, 100) \\ \cline{2-4}  & Kernel & Defining the algorithm & linear, rbf, poly, and sigmoid \\ \cline{2-4}  & Degree & The degree of the polynomial kernel & 1, 2, 3, 4, 5, 6 \\ \cline{2-4}  & Gamma & Kernel coefficient & scale, auto \\ \hline \multirow{4}{*}{Decision Tree (DT)} & Criterion & The function to measure the quality of a split & Gini and entropy \\ \cline{2-4}  & Max\_depth & The maximum depth of the tree & None, 2, 4, 6, 8, 10, or 12 \\ \hline \multirow{4}{*}{Random forest (RF)} & N\_estimators & Number of estimators & 10, 20, 50, 64, 100, 140, 200, and 256 \\ \cline{2-4}  & Min\_samples split & Minimal sample count necessary to split an internal node & 1, 2, 3, 6 \\ \cline{2-4}  & Min\_samples\_leaf & Minimum amount of samples at the tree’s leaves & 1, 6, and 10 \\ \cline{2-4}  & Max\_features & The number of features to consider for the best split & sqrt, log2, None \\ \cline{2-4}  & Criterion of trees & The function to measure the quality of a split & Gini and entropy \\ \cline{2-4}  & Max depth & The maximum depth of the tree & None, 2, 4, 5, 6, 7, 8, 16, or 30 \\ \hline \multirow{4}{*}{Naive Bayes (NB)} & Smoothing & Laplace smoothing technique helps tackle the problem of zero probability & \(\begin{array}{c}(1\times 10^{-1},1\times 10^{-2},1\times 10^{-3},1\times 10^{-4}, \\ 1\times 10^{-5},1\times 10^{-4},1\times 10^{-5},\\ 1\times 10^{-9})\end{array}\) & \(\begin{array}{c}(1\times 10^{-1},1\times 10^{-2},1\times 10^{-3},1\times 10^{-4},\\ 1\times 10^{-5},1\times 10^{-4},1\times 10^{-7},1\times 10^{-8},\\ 1\times 10^{-9})\end{array}\) \\ \hline \multirow{4}{*}{Extreme Gradient Boosting (MCB)} & Max depth & Maximum depth of the individual estimators/trees & 3, 4, 5, 6, 8, 10, 12, 15 \\ \cline{2-4}  & Gamma & Minimum loss reduction required to partition a leaf node of the tree & 0, 0.1, 0.2, 0.3, 0.4 \\ \cline{2-4}  & Colsample by tree & subsample ratio of columns when constructing each tree & 0.3, 0.4, 0.5, 0.7, 1 \\ \cline{2-4}  & Learning rate & Step size shrinkage used in update to prevents overfitting & 0.05, 0.10, 0.15, 0.20, 0.25, 0.3 \\ \cline{2-4}  & Min child weight & Minimum sum of instance weight needed in a child & 1, 3, 5, 7 \\ \hline \multirow{4}{*}{Category Boosting (CatBoost)} & Depth & Depth of the tree & 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 \\ \cline{2-4}  & The rate at which the model weights & & \\ are updated after working through each & & 0.01, 0.02, 0.03, 0.04, 0.009 \\ \cline{2-4}  & Learning rate & The maximum number of trees that can be built & 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 250, 500 \\ \cline{2-4}  & Hidden layer sizes & Number of hidden layers & (50,50,50), (50,100,50), (100, 30,10), \\ \cline{2-4}  & Activation & Activation function for the hidden layer & ‘identity’, ’logistic’, ‘tanh’, or ‘relu’. \\ \cline{2-4}  & Solver & The solver for weight optimization & \(\dagger\)bfgs’, ‘sgd’, or ‘adam’ \\ \hline \multirow{2}{*}{Alpha} & Alpha & Strength of the L2 regularization term & 0.0001, \(1\times 10^{-5}\), and 0.05 \\ \cline{2-4}  & Learning rate & Learning rate schedule for weight updates & ‘constant’, ‘invscaling’, ‘adaptive’ \\ \cline{2-4}  & Activation rate & & \\ \cline{2-4}  & Weight rate & & \\ \hline \hline \end{tabular}
\end{table} 
Table 4: The different values of models’ hyperparameters used in our study.