## 5 Results

Table 4: The best hyperparameter settings for particular algorithms and attribute subsets identified by tuning. \begin{table}
\begin{tabular}{l l l l l} \hline \hline
**Algorithm** & **Hyperparameter** & **Values** & **Default** \\ \hline \multirow{6}{*}{LR} & class\_weight & ’balanced’ & None \\  & solver & ’bbfgs’, ’liblinear’ & ’lbfgs’ \\  & penalty & ’l1’, ’l2’, ’elasticnet’ & ’l2’ \\  & c & 100, 10, 1.0, 0.1, 0.01 & 1 \\ \hline \multirow{6}{*}{SVC} & class\_weight & None, ’balanced’ & None \\  & c & 0.1, 1, 10, 100 & 1 \\  & gamma & ‘scale’, ’auto’ & ’scale’ \\  & kernel & ’linear’, ’rbf’ & ’rbf’ \\ \hline \multirow{6}{*}{RFC} & class\_weight & None, ’balanced’ & None \\  & n\_estimators & 25, 100, 250, 500 & 100 \\  & min\_samples\_split & 3, 7, 10, 13 & 2 \\  & max\_depth & None, 2, 3, 5 & None \\ \hline \multirow{6}{*}{GBC} & n\_estimators & 25, 100, 250, 500 & 100 \\  & min\_samples\_split & 3, 7, 10, 13 & 2 \\  & max\_depth & 1, 2, 3, 5 & 3 \\ \hline \multirow{6}{*}{KNC} & n\_neighbors & 2, 3, 5, 10, 15 & 5 \\  & leaf\_size & 20, 25, 30, 35 & 30 \\  & p & 1, 2 & 2 \\ \hline \multirow{6}{*}{MCPL} & hidden\_layer\_sizes & (100), (50), (50,50) & (100) \\  & activation & ‘tanh’, ’relu’ & ’relu’ \\ \cline{1-1}  & solver & ’sgd’, ’adam’ & ’adam’ \\ \cline{1-1}  & alpha & 0.0001, 0.05 & 0.0001 \\ \cline{1-1}  & learning\_rate & ’constant’, ’adaptive’ & ’constant’ \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Hyperparameters tuned for particular algorithms. The obtained PR and ROC curves are presented in Figures 4 and 5: each plot corresponds to a single algorithm and presents the curves for different attribute subsets. The values of the corresponding areas under these curves, summarizing the overall predictive performance, are presented in Tables 5 and 6. For easier comparison, the latter are also visualized by barplots in Figure 6 and 7. \begin{table}
\begin{tabular}{l l l l} \hline \hline
Figure 4: PR curves for particular algorithms and attribute subsets. (**a**) LR classifier; (**b**) SVC classifier; (**c**) RF classifier; (**d**) GB classifier; (**e**) KN classifier; (**f**) MLP classifier. Figure 5: ROC curves for particular algorithms and attribute subsets. (**a**) LR classifier; (**b**) SVC classifier; (**c**) RF classifier; (**d**) GB classifier; (**e**) KN classifier; (**f**) MLP classifier. Several conclusions can be drawn from the results obtained:  
* The best prediction corresponds to a PR AUC of about 0.5 and an ROC AUC of about 0.92;
* Depending on the choice of cutoff thresholds, model, operating points are possibly achieving the true-positive rate of more than 0.9 with the false-positive rate below 0.2 or both the recall and precision above 0.5;
* PR curves and PR AUC values reveal more substantial predictive performance differences than do ROC curves and ROC AUC values, which is to be expected for heavily imbalanced data; \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Model** & **S1** & **S2** & **S3** & **S4** & **S5** & **S6** & **S7** \\ \hline LR & 0.818 & 0.799 & 0.687 & 0.797 & 0.896 & 0.898 & 0.898 \\ SVC & 0.816 & 0.799 & 0.561 & 0.795 & 0.899 & 0.87 & 0.872 \\ RFC & 0.818 & 0.856 & 0.718 & 0.854 & 0.897 & 0.915 & 0.917 \\ GBC & 0.806 & 0.833 & 0.72 & 0.836 & 0.896 & 0.891 & 0.894 \\ KNC & 0.683 & 0.739 & 0.625 & 0.737 & 0.789 & 0.79 & 0.797 \\ MLPC & 0.816 & 0.797 & 0.677 & 0.789 & 0.897 & 0.888 & 0.885 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: ROC AUC values for particular algorithms and attribute subsets. Figure 6: PR AUC values for particular algorithms and attribute subsets.