# A Partially Interpretable Adaptive Softmax Regression for Credit Scoring
###### Abstract  
Credit scoring is a process of determining whether a borrower is successful or unsuccessful in repaying a loan using borrowers' qualitative and quantitative characteristics. In recent years, machine learning algorithms have become widely studied in the development of credit scoring models. Although efficiently classifying good and bad borrowers is a core objective of the credit scoring model, there is still a need for the model that can explain the relationship between input and output. In this work, we propose a novel partially interpretable adaptive softmax (PIA-Soft) regression model to achieve both state-of-the-art predictive performance and marginally interpretation between input and output. We augment softmax regression by neural networks to make it adaptive for each borrower. Our PIA-Soft model consists of two main components: linear (softmax regression) and non-linear (neural network). The linear part explains the fundamental relationship between input and output variables. The non-linear part serves to improve the prediction performance by identifying the non-linear relationship between features for each borrower. The experimental result on public benchmark datasets shows that our proposed model not only outperformed the machine learning baselines but also showed the explanations that logically related to the real-world. softmax regression; neural network; credit scoring application; decision making +
Footnote †: journal:  
networks to make its estimated probabilities adaptive for each class (see Figure 1). We first compute a linear transformation of input variables based on the softmax regression to obtain logits for each borrower. Secondly, we also perform a neural network (non-linear part) to augment logit of each class to make them adaptive for dealing with an imbalance problem. Finally, the summed linear and non-linear (output of neural network) transformations are fed into the softmax function to the probability of each class. The linear part partially explains the fundamental relationship between input and output variables, and the nonlinear part serves to improve the prediction performance by identifying the non-linear relationship between features for each borrower. The PIA-Soft architecture we propose is similar to the residual neural network model (ResNet), with the linear transformation acting as a residual block [12]. However, the advantage over ResNet architecture is that the PIA-Soft model can be partially explainable. To show achievement of the proposed model, we compare our model to high-performance machine learning benchmarks such as Logistic Regression, Random Forest, AdaBoost, XGBoost, Neural Network, LightGBM, Catboost, and TabNet [13; 14; 15; 16; 17; 18; 19; 20]. We apply our proposed model to over four benchmark real-world credit scoring datasets. The model performance on the test set is evaluated against three theoretical measures, an area under the curve (AUC), f-score, g-mean, and accuracy [21]. Our proposed model significantly outperformed machine learning models in terms of predictive performance. In order to evaluate the interpretation of PIA-Soft model, we compare our result to logistic regression because this model is the most popular white-boxing approach that is commonly used on credit scoring application. Here are some properties of logistic regression that make it a major benchmark--good predictive accuracy, high-level interpretability, and the modeling process is faster and easier [22]. Therefore, we can utilize it to verify the trustworthiness of our proposed model by comparing its unbiased estimated coefficients for input variables. Figure 1: Partially interpretable adaptive softmax (PIA-Soft) architecture: where \(x\). is an input, and \(p_{0}\) and \(p_{1}\). are the predicted probabilities of the first and second classes, respectively. In the end, the main contributions of this paper are included as follows:  
* To achieve high predictive accuracy, usually, model complexity is increased.