# Extreme Learning Machine Enhanced Gradient Boosting for Credit Scoring
## 5 Conclusions  
In this study, a supervised NN-based augmented GBDT-AugBoost-ELM is proposed to improve the performance of credit scoring. AugBoost-ELM is a variant of GBDT, which is step-wisely enhanced by the extreme learning machine to enhance the diversity of DTs in GBDT. ELM-based feature augmentation process not only provides robust augmented feature generation but also improves the efficiency of the feature augmentation process for GBDT compared with NN-based feature augmentation. The experimental results on four credit datasets show that AugBoost-ELM is an effective GBDT to improve the performance of credit scoring. Compared with NN-based feature supervised feature augmentation, ELM-based feature augmentation is not only beneficial to the performance of the boosting framework but also provides a way to accelerate the training process of supervised augmented GBDT. Though AugBoost-ELM improves the performance of credit scoring and provides an efficient way to augment features for the GBDT framework, there are some issues that need to be further tackled: (1) in this study, the NN architecture and ELM framework are optimized via grid search method, future work will focus on introducing some advanced hyper-parameters optimization algorithms such as Bayesian optimization method [46; 47], and evolutionary optimization strategies [48]. (2) The research of ML-based credit scoring is gradually marching towards the direction of large-scale credit scoring, in future work, some larger-scale datasets will be collected to further testify the effectiveness of AugBoost-ELM. (3) Classical GBDT is an inefficient training framework compared with XGBoost and LightGBM. Consequently, some efficient supervised and unsupervised feature augmentation methods are considered into the XGBoost and LightGBM [49] to further enhance the performance and improve the efficiency of credit scoring. Conceptualization and supervision, C.G. ; Investigation, Methodology, validation, writing-original draft, Y.Z. ; writing-review & editing, C.G. and Y.Z. All authors have read and agreed to the published version of the manuscript. This work was supported in part by the National Natural Science Foundation of China (71874027), in part by the Shanghai Planning Office of Philosophy and Social Science (2021 ECK001), and in part by the Shanghai Sailing Program (21 YF1415900). Not applicable. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Algorithm** & **Australian** & **German** & **Japanese** & **Taiwan** \\ \hline AugBoost-RP & 0.32 & 1.11 & 0.45 & 3.61 \\ AugBoost-PCA & 1.05 & 1.33 & 0.52 & 3.86 \\ AugBoost-NN & 245.21 & 280.78 & 300.89 & 3673.68 \\ AugBoost-NN (GPU) & 270.34 & 260.12 & 254.33 & 388.49 \\ AugBoost-ELM & 32.12 & 41.45 & 35.66 & 51.94 \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Training cost of various AugBoost-based models on the four credit datasets: all the training costs are averaged over 50 times 10-fold cross-validation, the unit of training cost is second, all the results are run on the experimental environment of CPU Intel 9700K with RAM 32 GB and GPU 1080 Ti; our implementation is based on python 3.5, gradient boosting framework are realized based on Scikit-learn 0.19.1, AugBoost-NN (GPU) is accelerated by a 1080 Ti GPU, which is implemented and tested in Keras with Tensorflow backend. +++ ==WARNING: Truncated because of repetitions==  
++  
* (29) [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON] A novel multi-stage ensemble model with enhanced outlier adaptation for credit scoring. _Expert Syst. Appl._**2021**, _165_, 113872. [CrossRef]
* (30) [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON] Dynamic ensemble classification for credit scoring using soft probability. _Appl. Soft Comput._**2018**, _65_, 139-151. [CrossRef]
* (31) [PERSON]; [PERSON]; [PERSON] A new hybrid ensemble model with voting-based outlier detection and balanced sampling for credit scoring. _Expert Syst. Appl._**2021**, _174_, 114744.