# A Partially Interpretable Adaptive Softmax Regression for Credit Scoring
## 4 Experimental Results
### Machine Learning Baselines and Hyperparameter Setting
For other evaluation metrics, neural network model with ADASYN sampling technique achieved the highest performance. In addition, our model achieved the similar performance compared to the state-of-the-art machine learning benchmarks on the Australian dataset, as shown in Table 4. CabBoost model with no sampling technique showed the best performance for AUC metric as well as this model achieved the highest performance with SMOTE sampling method for other evaluation metrics. Our model provides an improvement over the Logistic regression, Random forest, AdaBoost, Neural Network, and TabNet models by around 0.07 AUC, 0.002 accuracy, and 0.004 g-mean. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Model** & **Parameters** & **Search Space** \\ \hline \multirow{8}{*}{Random Forest} & max\_depth & (2, 8) \\  & min\_samples\_split & (1, 8) \\  & min\_samples\_leaf & (1, 8) \\  & criterion & \{’gini’,’entropy’\} \\  & bootstrap & [True, False] \\ \hline \multirow{2}{*}{AdaBoost} & learning\_rate & (0.1, 1) \\  & algorithm & \{’SAMME.R’, ’SAMME’\} \\ \hline \multirow{8}{*}{XGBoost} & min\_child\_weight & (1, 10) \\  & gamma & \{0, 0.1, 0.5, 0.8, 1] \\  & subsample & [0.5, 0.75, 0.9] \\  & colsample\_bytree & \{0.5, 0.6, 0.7, 0.8, 0.9, 1] \\  & max\_depth & [2, 8] \\  & learning\_rate & \{0.01, 0.1, 0.2, 0.3, 0.5] \\ \hline \multirow{8}{*}{LightGBM} & min\_child\_samples & (10, 60) \\  & reg\_alpha & \{0, 0.1, 0.5, 0.8, 1] \\  & subsample & \{0.5, 0.75, 0.9] \\  & colsample\_bytree & \{0.5, 0.6, 0.7, 0.8, 0.9, 1] \\  & max\_depth & (2, 8) \\  & learning\_rate & \{0.01, 0.1, 0.2, 0.3, 0.5] \\ \hline \multirow{8}{*}{CatBoost} & min\_child\_samples & (10, 60) \\  & subsample & \{0.5, 0.75, 0.9] \\  & subsample\_bytree & \{0.5, 0.6, 0.7, 0.8, 0.9, 1] \\  & max\_depth & (2, 8) \\  & learning\_rate & \{0.01, 0.1, 0.2, 0.3, 0.5] \\ \hline \multirow{8}{*}{TabNet} & n\_d & (4, 16) \\  & n\_a & (4, 16) \\  & mask\_type & \{’entmax’, ’sparsemax’\} \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Searching space of hyper-parameters. For the Taiwan dataset (see Table 5), CatBoost model achieved the highest performances, which are 0.753 AUC, 0.734 accuracy, 0.734 F-score, and 0.734 g-mean. Our proposed model showed the third best performance by achieving 0.744 AUC, 0.725 accuracy, 0.726 F-score, 0.726 g-mean. Since this dataset is balanced, we do not use the sampling techniques. Regarding the FICO dataset (see Table 6), our model achieved the best predictive performance for all evaluation metrics. Neural Network model with ROS sampling technique achieved the second best predictive performance on AUC metric. The logistic regression model with ROS sampling technique achieved the second best performance for other evaluation metrics. Our model improved the second best performance by around 0.008 AUC, 0.021 accuracy, 0.021 F-score, and 0.021 g-mean. In the end, our model succeeds the best predictive performance over most of the datasets. Therefore, this experiments provides evidence that our proposed PIA-Soft model equipped with a neural network works better than benchmark machine learning models on public credit scoring datasets. The next part of the experiments will show the interpretability of PIA-Soft model.