# Mmp1
## 2 Related Works
### Text-Based Feature Extraction Techniques  
Recently, many feature representation techniques have been suggested in order to train various machine-learning classification models such as naive Bayes (NB), support vector machine (SVM), and random forest (RF). Therefore, most of these techniques implement the vector space model (VSM) to represent an e-mail. In VSM, the \((i,j)\)-th value in a term-by-document matrix generally represents the occurrence of the \(i\)-th term within the \(j\)-th document, where this term can reflect an individual word or text unit [8]. For instance, the bag of words (BoW), the \(n\)-gram, and the term frequency inverse document frequency (TF-IDF) are the most popular algorithms used for feature representation. The authors of [9] used these feature techniques to transform incoming e-mails into a set of feature vectors. Then, these vectors were fed to evaluate different nearest neighbor classifiers, including k-NN, weighted KNN, and k-d tree, in order to classify e-mails as either spam or ham. In addition, [PERSON]. [10] used the term frequency technique to extract the features that are employed to train SVM for spam detection. Despite the success of the SVM model, other authors [11] have suggested the use of naive Bayes in the field of spam detection. However, despite the good results obtained through using such feature representation techniques, they have some limitations. The authors of [12] stated that the BoW and the TF-IDF techniques lead to overfitting classifiers because they have a large sparse feature space that is caused by their large vocabulary. In addition, [PERSON]. [13] confirmed that the higher dimensionality of features is one of the biggest issues of spam detection. Thus, several dimensionality reduction techniques have been suggested in order to overcome the above problem. Many studies suggested the use of a singular value decomposition (SVD)-based approach for feature reduction. There are two well-known SVD-based techniques: principal component analysis (PCA) and latent semantic analysis (LSA). They are used to convert the original space to a new lower-dimensional feature space. The authors of [13] proposed the use of PCA and LSA for the pre-processing step of the e-mails, while using the TF-IDF for building the feature vectors which were fed to a linear SVM classifier, called sequential minimal optimization (SMO). Similarly, [PERSON]. [14] provided a new model using a classifier based on PCA document reconstruction (PCADR) which was compared with the popular SMO classifier. The authors of [15] declared that, because LSA is a term co-occurrence-based technique, it does not work for noisy data. In addition, [PERSON]. [12] stated that LSA does not capture word order although it achieved good performance. In recent years, many techniques have been adopted for spam e-mail detection such as PV-DBOW. This technique takes into consideration the context of e-mails. The authors of [16] proposed a new architecture to detect spam e-mails using the PV-DBOW model. This technique was used to extract features from e-mails that were fed to various classification models in order to distinguish between spam and ham. The authors of [17] compared various deep learning techniques to the classical machine learning classifiers for detecting spam e-mails. In addition, different word embedding methods were used in order to generate a dense feature vector from an e-mail. The authors of [17] proposed the DeepSpamNet architecture that uses the CNN and the LSTM methods. In the work of [18], various machine learning techniques such as support vector machine, naive Bayes, decision tree, random forest, and multilayer perceptron, which were optimized using the PSO (particle Swarm Optimization) and the GA (genetic algorithm), were adopted in order to detect spam e-mails. The experiments were conducted on seven datasets to select the most suitable model. The pretrained transformer model BERT (bidirectional encoder representations from transformers) was used in the work of [19].