# A Novel Ensemble Belief Rule-Based Model for Online Payment Fraud Detection
## 3 Preliminaries
### Feature Selection  
In fraud detection datasets, there are typically numerous attributes, but not all of them are equally relevant for the targeted task. For instance, certain attributes such as transaction amount and user age prove to be critical in detecting fraudulent transactions as abnormal transaction amounts suggest fraudulent behavior and some age groups are more susceptible to becoming targets of fraud. Conversely, attributes like user gender and transaction time may be less important for fraud detection. Therefore, selecting the most important attributes is a crucial step as it can significantly reduce the complexity of the model. Especially for BRB models, it helps avoid the combinatorial explosion problem during the BRB modeling process. In this work, XGBoost feature importance analysis is adopted as the feature selection method. Extreme gradient boosting [47] is an optimized gradient boosting implementation that is known for its efficiency, flexibility and state-of-the-art performance in many machine learning problems, such as classification and regression. During the past decades, it has become a widely used and highly effective machine learning tool. In this section, the mechanism of how XGBoost is used as a feature selection technique is briefly introduced as follows. The XGBoost algorithm is an ensemble learning method composed of decision trees. Take a dataset with input samples set at \(X=\{x_{1},x_{2},\ldots,x_{n}\}\) and corresponding labels set at \(Y=\{y_{1},y_{2},\ldots,y_{n}\}\). The dataset can be described as \(DS=\left\{(x_{i},y_{i}):i=1\ldots n,x_{i}\in\mathbb{R}^{M},y_{i}\in\mathbb{R}\right\}\). The output of XGBoost \(\hat{y}_{i}\) is defined as follows. \[\hat{y}_{i}=\sum_{k=1}^{K}f_{k}(x_{i}),f_{k}\in F \tag{9}\]  
where \(\hat{y}_{i}\) is the prediction value of input sample \(x_{i}\): \(f_{k}\) denotes the \(k\)-th decision tree in the tree space \(F\). \(F=\{f_{1},f_{2},\ldots,f_{K},\}\), where \(f_{k}:X\rightarrow\mathbb{R}\), and \(K\) is the total number of decision trees. The objective function \(Obj:\mathbb{R}\times Y\rightarrow\mathbb{R}\) is described in Equation (9). The goal is to minimize the function \(Obj\) in Equation (10). \[Obj=\sum_{i=1}^{n}l(\hat{y}_{i},y_{i})+\sum_{k=1}^{K}\Omega(f_{k}) \tag{10}\]  
where \(l(\hat{y}_{i},y_{i})\) is the loss between the \(i\)-th input sample's prediction value and real value and \(\Omega:F\rightarrow\mathbb{R}\) denotes the penalty parameters which is used for controlling the complexity of the decision tree in order to avoid overfitting. In the XGBoost algorithm, each decision tree is generated iteratively and the \(t\)-th decision tree is created based on fitting the loss between the sum of the outputs of the existing decision trees in the \((t-1)\)-th iteration and the true values. In this process, the goal is to minimize the function \(L^{(t)}\), which is illustrated in Equation (11)  
\[L^{(t)}=\sum_{i=1}^{n}\left[l\left(y_{i},\hat{y}_{i}^{t-1}+f_{t}(x_{i})\right) \right]+\Omega(f_{t}) \tag{11}\]  
where \(\hat{y}_{i}^{t-1}\) denotes the sum of existing decision trees' prediction of the \(i\)-th instance at the \((t-1)\)-th iteration. Each tree in XGBoost consists of several nodes and each node selects the attribute with the highest information gain at the current position from all the available attributes to serve as the node. XGBoost typically measures the importance of each attribute by calculating itsinformation gain, which is then used for feature selection. First, calculating the number of times each attribute serves as a node across all trees. \[Weight(feature_{i})=\sum_{k=1}^{K}Count_{k}(feature_{i}) \tag{12}\]  
where \(Weight(feature_{i})\) denotes the number of times the \(i\)-th attribute serves as a node across all trees. \(Count_{k}\) is the times the i-th attribute serves as a node in the \(k\)-th tree. After calculating the formula above, the next step is to calculate the average information gain of each attribute. The calculation formula is as follows.