# Comparison of Profit-Based Multi-Objective Approaches for Feature Selection in Credit Scoring
## 3 Methods
### Non-Dominated Binary Grasshopper Optimization Algorithm (NSBGOA)
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Objective** & **Optimization** & **Function** \\ \hline Expected maximum & maximize & \(EMP=\int_{0}^{1}(b*\pi_{0}\text{F}_{0}-ROI*\pi_{1}\text{F}_{1})*\text{g}(b)\; \text{d}b\) \\ \hline Number of features & & \\ (cardinality) & maximize & \(-n/j\) \\ \hline Ease of explanation & & \\ per feature set & maximize & \(c=\sum_{p=1}^{j}individual variable\ value_{p}/j\) \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Objectives. ## 5 Results

The mean hypervolumes of the final populations from five runs of the NSGA-III, NSGA-II, and NSBGOA algorithms are given in Figure 1. Further, Figure 2 shows the mean computational time from five runs of each algorithm on a computer with 8 GB RAM and an Intel(R) Core(TM) i5-7200U CPU 2.50 GHz, while the objective values of the feature set with max EMP are shown in Figure 3. For a single run of each algorithm, the distribution of points in the outputs of the multi-objective algorithms is shown with scatter plots in Figure 4. For the same run, a statistical summary of the final populations is given in Table 3, and the three algorithms trained to optimize a single objective have their results given in Table 4. The n



Figure 4: Scatter plots of final population. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Algorithm** & **Number of Features** & **Objective** & **Value** \\ \hline \hline \multirow{2}{*}{Lasso} & \multirow{2}{*}{20} & emp & \(0.1\) \\  & & cardinality & \(-1\) \\  & & affordability & \(0.25\) \\ \hline \multirow{2}{*}{All features with Logistic Regression} & \multirow{2}{*}{20} & emp & \(0.1\) \\  & & cardinality & \(-1\) \\  & & affordability & \(0.7\) \\ \hline \multirow{2}{*}{All features with Neural Network} & \multirow{2}{*}{20} & emp & \(0.103\) \\  & & cardinality & \(-1\) \\  & & affordability & \(0.7\) \\ \hline \multirow{2}{*}{All features with Linear Support Vector Machine} & \multirow{2}{*}{20} & emp & \(0.098\) \\  & & cardinality & \(-1\) \\  & & affordability & \(0.7\) \\ \hline \multirow{2}{*}{GA with Logistic Regression} & \multirow{2}{*}{17} & emp & \(0.099\) \\  & & cardinality & \(-0.85\) \\  & & affordability & \(0.6\) \\ \hline \multirow{2}{*}{GA with Neural Network} & \multirow{2}{*}{17} & emp & \(0.103\) \\  & & cardinality & \(-0.85\) \\  & & affordability & \(0.6\) \\ \hline \multirow{2}{*}{GA with Linear Support Vector Machine} & \multirow{2}{*}{13} & emp & \(0.103\) \\  & & cardinality & \(-0.85\) \\  & & affordability & \(0.6\) \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Single-objective results. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Algorithm** & \ \begin{tabular}{c} **Mean \# of** \\ **Features/Solution** \\ \end{tabular}  & **Objective** & **Min** & **Max** & **SD** & **Mean** & \begin{tabular}{c} **Nadir** \\ **Point** \\ \end{tabular}  \\ \hline NSGA-II with & & emp & \(0.083\) & \(0.093\) & \(0.087\) & \(0.087\) & \(0.093\) \\ Logistic Regression & \(11.8\) & cardinality & \(-0.8\) & \(-0.4\) & \(-0.59\) & \(-0.59\) & \(-0.4\) \\  & & ease of explanation & \(0.15\) & \(0.5\) &



## 6 Discussion

### Base Classifier

The performance of three base classifiers, namely logistic regression, neural network classifier, and linear support vector machine, was compared. To achieve this, each multi-objective optimization (MOO) algorithm was evaluated with the three base classifiers in turn. The neural network classifier gave the smallest mean number of features per solution (Table 3), with the best EMP values across all multi-objective methods. It is possible that the neural network classifier's ability to consider non-linear interactions in the data leads to better performance with fewer features. Additionally, this classifier required the greatest computational time (Figure 2), regardless of the MOO algorithm applied. This is to be expected as the neural network method has a higher computation complexity than the other two classifiers.