# A Novel Ensemble Belief Rule-Based Model for Online Payment Fraud Detection
## 5 Experimental Study
### Modeling Steps and Classification Results of Ensemble BRB  
This section presents several weak BRBs' results after model selection to demonstrate the experimental effectiveness of the corresponding steps. Step 1. Data sampling. As mentioned in Section 4, for this fraud detection task, we employed an innovative data sampling approach. We retained all the samples of fraud transactions in the dataset and only conducted bootstrap sampling on the normal transaction samples. The results after the data sampling of Dataset #1 are illustrated in Figure 8.  
Step 2. Feature selection. During the feature selection process, we employed the XGBoost feature importance analysis method to filter the most important two features of these two datasets. The example results of feature selection in Dataset #1 are presented in Table 3.  
Due to the sensitive nature of the dataset, the attributes are presented in an anonymized form, possibly representing processed transaction timestamps, dates and other features. The feature selection results reveal certain characteristics--the selected features demonstrate diversity, while some features like V14 and V10 appear with notably higher frequency than others. This result is reasonable since one of the most important aspects of ensemble learning is achieving a balance between diversity and performance. When the sampling rate is too high, there is excessive overlap between different datasets, resulting in a loss of diversity. Consequently, the attributes selected from each dataset are likely to be identical, meaning that these weak BRBs would likely produce similar outputs for the same sample, \begin{table}
\begin{tabular}{c c c} \hline \hline
**Dataset** & **Feature 1** & **Feature 2** \\ \hline Sub-dataset 1 & V10 & V14 \\ Sub-dataset 2 & V14 & V7 \\ Sub-dataset 3 & V14 & V10 \\ Sub-dataset 4 & V10 & V17 \\ Sub-dataset 5 & V10 & V14 \\ Sub-dataset 6 & V12 & V14 \\ Sub-dataset 7 & V16 & V10 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Feature selection results of weak BRBs in Dataset #1.  
defeating the purpose of ensemble learning. Conversely, if the sampling rate is too low, too much information is lost, leading to the poor performance of individual weak BRBs, which also results in unsatisfactory ensemble results. Figures 3 and 4 illustrate this point with a simple example. Step 3. Construction of weak BRBs. Based on the sub-dataset after feature selection in Step 2, the weak BRBs are constructed. The construction process of each weak BRB typically includes initial parameter settings, primarily involving the initialization of belief distribution, attribute weights, rule weights, etc. These parameters are usually determined based on expert knowledge. In the absence of specific research, attribute weights and rule weights are typically set to 1. The classification results of the weak BRBs after construction are shown in Tables 4 and 5. Figure 8: Data distribution after data sampling in Dataset #1. Step 4. Model selection. After step 3, multiple weak BRBs are constructed. Since there is a degree of randomness involved in building weak BRBs, a model selection process is required for all the weak BRBs. Based on their performance in the test set, such as their F1 score, the models with relatively good scores are selected to proceed to the next step. Step 5. Combining weak BRBs. After selecting the weak BRBs in step 4, their outputs are combined. In this paper, we employ a voting method to combine the outputs of the weak BRBs. As can be seen from Figure 8, the data sampled using the improved sampling method exhibit a lower degree of imbalance while still preserving most of the original data distribution characteristics. Moreover, according to the feature selection results shown in Table 3, the sampled data exhibit the following two main features. (1) Different weak BRBs select different features. (2) Some features appear more frequently than others. These two points indicate that the improved sampling method reaches a balance between diversity and performance.