# Phish Responder: A Hybrid Machine Learning Approach to Detect Phishing and Spam Emails
## 5 Evaluation and Discussion
### Comparison  
Phish Responder has been compared with two solutions (see Table 8). Figure 11: Individual email option. Figure 10: MLP model for numerical-based datasets. The accuracy and precision have been measured in this research so the effectiveness of Phish Responder can be evaluated and compared with the solutions in Table 8. As seen in Table 8, Phish Responder can achieve high accuracies and precisions, thus illustrating its efficiency in identifying phishing and spam emails. Over five runs, the LSTM model achieved an average accuracy of 99% and the MLP model achieved an average accuracy of 94%. Phish Responder's LSTM model with the Spam Assassin dataset can achieve a higher accuracy than the techniques in Table 8. Additionally, the MLP model produces a similar accuracy to the research outlined in Table 8. Therefore, when compared with existing research, it is confirmed that Phish Responder produces suitable accuracies and precisions for phishing and spam email detection. Alongside accuracy and precision, Phish Responder can identify phishing and spam emails in a timely manner as it can be used via the command prompt. This provides users with a quick identification of phishing and spam emails within a dataset and a quick extraction of features from an individual email. It is not known whether the existing research, presented in Table 8, has developed its techniques for the command prompt; however, [PERSON] et al. [5] provide a distinguishable feature which is real-time classification of emails--a feature neither included in Phish Responder nor [PERSON], [PERSON] and [PERSON] [6]. [PERSON], [PERSON] and [PERSON] [6] provided a comparison of NLP and ML algorithms to determine the best combination for the detection of phishing and spam emails; they achieved at least 90% accuracy when using TF-IDF. TF-IDF is a technique that Phish Responder uses, but deep learning is used, as opposed to machine learning. Both [PERSON], [PERSON] and [PERSON] [6] and [PERSON] et al. [5] state that deep learning would be considered as future work in their research. [PERSON] et al. [5] also used a combination of NLP and ML algorithms, achieving respectable accuracies and precisions, but their focus was on spam emails. [PERSON], [PERSON] and [PERSON] [6] concentrated on the detection of phishing emails. Phish Responder differs as it has been developed to identify phishing and spam emails and both spam and phishing datasets were used to train the LSTM and MLP models. Phish Responder was trained using open-source datasets, mainly from Kaggle. [PERSON] et al. [5] also used a dataset from \begin{table}
\begin{tabular}{c c c} \hline \hline
**[PERSON], [PERSON] and [PERSON] [6]** & **[PERSON] et al. [5]** & **Phish Responder** \\ \hline NLP and ML for phishing email detection. Uses TF-IDF with Logistic Regression, Decision Tree, Random Forest, GBT and Naive Bayes. Uses the Enron email corpus. & Classify spam emails using NLP techniques and ML algorithms such as Naive Bayes and SVM. Uses the SMS Spam Collection Dataset and the Enron Spam dataset. & NLP and deep learning for phishing and spam email detection. Uses LSTM for text based and MLP for numerical based datasets. Provides an individual email element to extract interesting features. Uses the Spam Assassin from the Email Spam Dataset from Kaggle [25] and the Email Spam Classification Dataset from Kaggle [28]. \\ \hline \hline \multicolumn{3}{c}{Accuracies for balanced datasets using TF-IDF: LR—92.48%, DT—90.17%, RF—93.41%, GBT—91.47%, NB—92.77%. Accuracies for imbalanced datasets using TF-IDF: LR—98.41%, DT—95.92%, RF—94.43%, GBT—97.28%, NB—92.05%.} \\ \hline \hline \multicolumn{3}{c}{Precisions for balanced datasets using TF-IDF: LR—0.9110, DT—0.9063, RF—0.9743, GBT—0.9301, NB—0.9652. Precisions for imbalanced datasets using TF-IDF: LR—0.9031, DT—0.8655, RF—1, GBT—0.9032, NB—0.} \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Research comparison.