## 4 Experiments
### Evaluation Metrics  
The most common metrics used to measure classification performances are accuracy, precision, recall, and the F1-score. In class imbalance problems, because the accuracy rate does not reflect the overall situation, we employ the true positive rate (TPR), false positive rate (FPR), precision, F1-score, and G-mean to measure the performance of the proposed spam detection method. We apply a confusion matrix to calculate these indicators.  
As shown in Table 3, each row represents a class while each column represents the predicted class. TP, FP, FN and TN represent true positives, false positives, false negatives, and true negatives, respectively. The performance measures can be calculated as follows:  
\(\bullet\) True positive rate (TPR)  
The true positive rate is known as recall rate, which indicates the ratio of the correct classification of positive samples.  
\[TPR=\frac{TP}{TP+FN} \tag{7}\]  
\(\bullet\) False positive rate (FPR)  
The false positive rate indicates the ratio that classifies the negative samples into positive samples. In this paper, FPR represents the proportion of classifying a majority class into a minority class.  
\[FPR=\frac{FP}{FP+TN} \tag{8}\]  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Feature** & **Description** \\ \hline account\_age & The age (days) of an account since its creation until the time of sending the most recent tweet \\ no\_follower & The number of followers of this twitter user \\ no\_following & The number of followings/friends of this twitter user \\ no\_userfavourites & The number of favorites this twitter user received \\ no\_lists & The number of lists this twitter user added \\ no\_tweets & The number of tweets this twitter user sent \\ no\_retweets & The number of retweets this tweet \\ no\_hashtag & The number of hashtags included in this tweet \\ no\_usermention & The number of users mentions included in this tweet \\ no\_uris & The number of URLs included in this tweet \\ no\_char & The number of characters in this tweet \\ no\_digits & The number of digits in this tweet \\ \hline \hline \end{tabular}
\end{table}
Table 2: Twitter spam dataset.  
\begin{table}
\begin{tabular}{c c c} \hline \hline  & **Predicted Spam** & **Predicted Non-Spam** \\ \hline Actual Spam & True positive (TP) & False negative (FN) \\ Actual non-spam & False positive (FP) & True negative (TN) \\ \hline \hline \end{tabular}
\end{table}
Table 3: Confusion matrix.  
Precision represents the ratio of correctly predicted positive samples to total predicted positive samples.  
\[Precision=\frac{TP}{TP+FP} \tag{9}\]  
* F1-score  
The F1-score is a weighted average of precision and recall. It is an important performance metric to evaluate the overall performance of our method.  
\[F1-score=2*\frac{Precision*TPR}{Precision+TPR} \tag{10}\]  
* G-mean  
G-mean measure is to evaluate the degree of inductive bias according to the ratio of positive precision to negative precision. The higher G-mean represents that the classifier has better classification performance in both majority and minority classes.  
\[G-mean=\sqrt{\frac{TP}{TP+FN}*\frac{TN}{TN+FP}} \tag{11}\]  
* Kappa  
Kappa is an important index to measure the classification performance on imbalanced datasets, which represents the proportion of error reduction between classification and completely random classification. It measures the consistency between classifier and target distribution hypothesis.  
\[Kappa=\frac{p_{o}-p_{e}}{1-p_{e}} \tag{12}\]  
where  
\[p_{o}=\frac{TP+TN}{TP+FP+TN+FN} \tag{13}\]  
\[p_{e}=\frac{(TP+FP)*(TP+FN)+(FN+TN)*(FP+TN)}{(TP+FP+TN+FN)*(TP+FP+TN+FN)} \tag{14}\]