## 4 Experiments
### Results and Discussion
#### 4.4.2 Comparisons with Class Imbalance Methods  
On the basis of the comparisons in the previous section of our proposed algorithm with conventional machine learning algorithms, we further compare our algorithm with more advanced algorithms regarding their abilities to solve the class imbalance problem. Section 2.2 mentions that ensemble learning can effectively improve the performance when faced with class imbalance problems. Therefore, we add ensemble learning algorithms that use majority voting as an ensemble strategy to the comparison. CSDNN is an algorithm based on cost sensitivity learning as discussed in Section 3.2, while AdaCost, MetaCost, and WSNN are all improved algorithms based on cost sensitive learning. Table 5 presents a comparison of the above algorithms in terms of TPR, FPR, precision, F1-score, and Kappa.  
As shown in Table 5, the performances of these improved methods are significantly better than those of the conventional machine learning algorithms. For example, the TPR value in Table 5 fluctuates less than that in Table 4 and the mean value is higher. The same phenomenon occurs for G-mean, with an average of 0.78 in this experiment, but an average of 0.59 in the previous experiment. These results show that the methods in this experiment provide improvements when faced with class imbalance problems.  
Figure 5 intuitively illustrates an experimental comparison between our proposed method and the improved methods. Considering the MetaCost method as an example, the TPR of this method is 0.69, which is very close to the 0.70 achieved by our method, but its accuracy, F1-score, and Kappa are 0.3, 0.19, and 0.22 lower than those of our method. This phenomenon indicates that although the method obtains a better positive sample recognition rate, it also misclassifies a large number of  
Figure 4: Spam detection performance compared with different machine learning algorithms.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Method** & **TPR** & **FPR** & **Precision** & **F1-Score** & **G-Mean** & **Kappa** \\ \hline Ensemble Learning & 0.69 & 0.04 & 0.63 & 0.66 & 0.81 & 0.62 \\ CSDNN & 0.54 & 0.05 & 0.52 & 0.53 & 0.71 & 0.48 \\ AdaCost & 0.67 & 0.09 & 0.41 & 0.51 & 0.78 & 0.45 \\ MetaCost & 0.69 & 0.10 & 0.40 & 0.51 & 0.77 & 0.45 \\ WSNN & 0.63 & 0.08 & 0.44 & 0.52 & 0.76 & 0.46 \\ Our approach & 0.70 & 0.03 & 0.70 & 0.70 & 0.82 & 0.67 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Classification results of different class imbalance methods.  
negative samples. In the same way, the ensemble learning method achieved good results in this group of experiments, but its accuracy and F1-score are 7% and 4% lower than that of our method, and its Kappa is 5% lower. Therefore, our method yields good performances as compared with those of other improved methods.  
In addition, it is worth noting that CSDNN, AdaCost, MetaCost, and our method are all improved by cost-sensitive learning, but our method adopts the integrated learning framework based on stacking, and the data processing of the basic module effectively improves the classification performance of the overall algorithm.