# Extreme Learning Machine Enhanced Gradient Boosting for Credit Scoring
## 4 Experimental Results  
To test the effectiveness of AugBoost-ELM, we first visualize the ROC curves of credit scoring models for comparison. ROC curve, also known as receiver operating characteristic curve, is a graphical measurement that reflects the sensitivity and specificity of credit scoring models under the different thresholds of predictive probability. The \(x\)-axis represents the value of false positive rate (\(FPR\)) and the \(y\)-axis is the value of true positive rate (\(TPR\)), where \(TPR\) can be calculated as:  
\[TPR=\frac{TP}{TP+FN}. \tag{13}\]  
FPR is defined as:  
\[FPR=\frac{FP}{FP+TN}. \tag{14}\]  
In Equations (13) and (14), the prediction results can be classified into four groups: TP, FP, TN, FN, which can be viewed from Table 3.  
Where TP is the number of accurate classified samples whose label is \"bad\"; FP counts the number of samples that are labeled as \"good\" while the prediction results are \"bad\"; FN calculates the number of samples whose label is \"bad\" and predicted as \"good\"; TN represents the number of \"good\" applicants that are correctly classified. The larger the area under the ROC curve implies the better performance a credit scoring algorithm is. To testify the effectiveness of the feature enhancement mechanism for boosting framework, some baseline models are first selected for preliminary study. These models include statistical-based algorithms such as LR and LDA, ML-based individual classifiers such as DT, KNN, SVM, and NN, bagging-type ensemble method RF and boosting-class ensemble approach GBDT. Figure 2 shows the ROC curves of various credit scoring models on the credit datasets. Figure 2a represents the ROC curves of credit scoring models for the Australian dataset; Figure 2b is the ROC curves of credit scoring models for the German dataset; Figure 2c denotes the ROC curves of credit scoring models for the Japanese dataset; Figure 2d illustrates the ROC curves of credit scoring models for the Taiwan dataset. All the ROC curves are an average based on 50 times repeated 10-fold cross-validation. As is shown in Figure 2a, on the Australian dataset, LR and LDA, KNN gets the smallest area under the ROC curve while AugBoost-ELM gets the largest area under the \begin{table}
\begin{tabular}{c c c c} \hline \hline  & \multicolumn{3}{c}{**Label**} \\ \hline  & & **Bad** & **Good** \\ \hline Prediction & Bad & TP & FP \\ \cline{2-4}  & Good & FN & TN \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Confusion matrix of prediction results. Figure 2: ROCs comparison of benchmark credit scoring models for credit datasets: RG represents the curve of random guess. (**a**) Australian, (**b**) German, (**c**) Japanese, (**d**) Taiwan. ROC curve, which demonstrates that AugBoost-ELM is the best algorithm to predict the PD for Australian compared with other baseline models. Compared with KNN, though DT improves the predictive performance, its ROC curve is worse than those of other credit scoring models, indicating that the single DT is not a good solution for accurate credit scoring. Compared with ML-based credit scoring models, statistical-based algorithms LR and LDA get the larger areas under ROC curves, providing evidence that why LR and LDA are popularized by industrial application. Compared with ML-based individual classifiers, the areas under ROC curves of RF and GBDT are significantly larger, which suggests that ensemble multiple weak learners into a stronger one is a good strategy to improve the performance of credit scoring. As can be viewed from Figure 2c, on the Japanese dataset, the largest area under the ROC curve of AugBoost-ELM implies AugBoost-ELM is the best algorithm for the credit scoring of the Japanese dataset. In addition, SVM gets the smallest area under the ROC curve. Though the ROC curves of other ML-based credit scoring algorithms such as DT, KNN, NN show better predictive ability than SVM, their performance on predicting the PD is worse than that of ensemble learning approaches.