## 3 Problem Description and Methodology
### The Proposed Ensemble Learning Framework
#### 3.2.2 Combining Module
It assumes that an asymmetric misclassification cost exists between classes, defined in the form of a cost matrix, as shown in Table 1.  
The typical form of cost sensitive learning is to use a cost matrix such as the one shown in Table 1. A cost \(\Upsilon(p,q)\) is used to denote the cost of misclassifying an instance belonging to class p into a different class q. In spam detection, we regard spam as positive samples and non-spam as negative samples. Therefore, the misclassification cost \(\Upsilon(p,q)\) belonging to the minority class is higher than that belonging to the majority class \(\Upsilon(q,p)\). Using feedback from the base classifier performances, we use the misclassification ratio of the minority class in the base modules as the misclassification cost of the positive samples and set the misclassification cost of the majority class to one. The diagonal elements of the cost matrix, such as \(\Upsilon(p,p)\) and \(\Upsilon(q,q)\), represent correct predictions, and the misclassification cost is equal to zero \(\Upsilon(p,p)=\Upsilon(q,q)=0\)[32]. According to the minimum expected cost principle, the goal of cost sensitive classification is to train the classifier so that it classifies training samples into the class that has the minimum expected cost. Therefore, the classifier we are training provides a class decision for a sample. The expected risk \(R(p|x)\) with a sample \(x(x\in X)\) and \(i\) as the output classification can be expressed as follows: \begin{table}
\begin{tabular}{c c c} \hline \hline  & **Predicted Positive** & **Predicted Negative** \\ \hline True positive & \(\Upsilon(p,p)\) & \(\Upsilon(p,q)\) \\ True negative & \(\Upsilon(q,p)\) & \(\Upsilon(q,q)\) \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Cost matrix for binary classification. \[R(p|x)=\sum_{q=1}^{K}P(q|x)\,\Upsilon(p,q) \tag{3}\]  
where \(P(q|x)\) represents the posterior probability that a given sample \(x\) will be classified as class q in a dataset of \(K\) classes. On the basis of the Bayesian decision theory, an ideal classifier makes a final decision by calculating the expected risk of each sample classification and predicts the label that achieves the minimum expected risk:  
\[\underset{p}{\text{argmin}}\ R(p|x)=\underset{p}{\text{argmin}}\ E[\Upsilon] \tag{4}\]  
For a sample \(x^{(i)}\) and its corresponding label \(y^{(i)}\), the empirical risk can be expressed as:  
\[\overset{\wedge}{R}_{l}=E[l]=l(y^{(i)},y^{(i)},\Upsilon) \tag{5}\]  
where \(l(\cdot)\) represents a loss function, such as mean square error (MSE) or the cross-entropy loss function, and N is the total number of data samples. As described in Section 3, the goal of cost-sensitive learning is to minimize the overall cost of the training dataset (e.g., the Bayesian conditional risk). The misclassification cost can be regarded as a penalty factor introduced during the classifier training process (or in some cases in the forecasting step) to improve the importance of classes that are difficult to classify (such as the spam class). By imposing larger penalties for errors on a given class, we force the classifier training process (intended to minimize overall cost) to focus on instances from the given distribution. In this study, we use cost-sensitive modified cross entropy as the loss function during classifier training. This paper mainly focuses on spam detection, and therefore we pay more attention to the spam class than to the non-spam class. Thus, \(\Upsilon(p,q)\) should be larger than \(\Upsilon(q,p)\), forcing the error classification cost for spam samples to be higher. The overall error of the cost-sensitive DNN can be formulized as follows:  
\[loss=-\frac{1}{N}\!\sum_{n=1}^{N}\left[y_{n}*\log(P(y=j|x,\theta)*\Upsilon(p, q))\right.+(1-y_{n})*\log(1-P(y=j|x,\theta)*\Upsilon(q,p))] \tag{6}\]  
The backpropagation algorithm is the most common method for optimizing the DNN parameters and is essentially a gradient descent function.