How to Build a Spam Classifier in 10 Steps | Towards Data Science

If you’re just starting out in Machine Learning, chances are you’ll be undertaking a classification project. As a beginner, I built an SMS spam classifier but did a ton of research to know where to start. In this article, I’ll walk you through my project in 10 steps to make it easier for you to build your first spam classifier using Tf-IDF Vectorizer, and the Naïve Bayes model!
Our SMS text messages dataset has 5 columns if you read it in pandas: v1 (containing the class labels ham/spam for each text message), v2 (containing the text messages themselves), and three Unnamed columns which have no use. We’ll rename the v1 and v2 columns to class_label and message respectively while getting rid of the rest of the columns.
import pandas as pddf = pd.read_csv(r'spam.csv',encoding='ISO-8859-1')df.rename(columns = {'v1':'class_label', 'v2':'message'}, inplace = True)df.drop(['Unnamed: 2', 'Unnamed: 3', 'Unnamed: 4'], axis = 1, inplace = True)df
Check out the fact that ‘5572 rows x 2 columns’ means that our dataset has 5572 text messages!
It’s a good idea to carry out some Exploratory Data Analysis (EDA) in a classification problem to visualize, get some information out of, or find any issues with your data before you start working with it. We’ll look at how many spam/ham messages we have and create a bar chart for it.
#exploring the datasetdf['class_label'].value_counts()
Our dataset has 4825 ham messages and 747 spam messages. This is an imbalanced dataset; the number of ham messages is much higher than those of spam! This can potentially cause our model to be biased. To fix this, we could resample our data to get an equal number of spam/ham messages.
To generate our bar chart, we use NumPy and pyplot from Matplotlib.
For my project, I generated word clouds of the most frequently occurring words in my spam messages.
First, we’ll filter out all the spam messages from our dataset. df_spam is a DataFrame that contains only spam messages.
df_spam = df[df.class_label=='spam']df_spam
Next, we’ll convert our DataFrame to a list, where every element of that list will be a spam message. Then, we’ll join each element of our list into one big string of spam messages. The lowercase form of that string is the required format needed for our word cloud creation.
spam_list= df_spam['message'].tolist()filtered_spam = filtered_spam.lower()
Finally, we’ll import the relevant libraries and pass in our string as a parameter:
import osfrom wordcloud import WordCloudfrom PIL import Imagecomment_mask = np.array(Image.open("comment.png"))#create and generate a word cloud imagewordcloud = WordCloud(max_font_size = 160, margin=0, mask = comment_mask, background_color = "white", colormap="Reds").generate(filtered_spam)
After displaying it:
Pretty cool, huh? The most common words in spam messages in our dataset are ‘free,’ ‘call now,’ ‘to claim,’ ‘have won,’ etc.
For this word cloud, we needed the Pillow library only because I’ve used masking to create that nice speech bubble shape. If you want it in square form, omit the mask parameter.
Similarly, for ham messages:
To handle imbalanced data, you have a variety of options. I got a pretty good f-measure in my project even with unsampled data, but if you want to resample, see this.
First, let’s convert our class labels from string to numeric form:
df['class_label'] = df['class_label'].apply(lambda x: 1 if x == 'spam' else 0)
In Machine Learning, we usually split our data into two subsets — train and test. We feed the train set along with the known output values for it (in this case, 0 or 1 corresponding to spam or ham) to our model so that it learns the patterns in our data. Then we use the test set to get the model’s predicted labels on this subset. Let’s see how to split our data.
First, we import the relevant module from the sklearn library:
from sklearn.model_selection import train_test_split
And then we make the split:
x_train, x_test, y_train, y_test = train_test_split(df['message'], df['class_label'], test_size = 0.3, random_state = 0)
Let’s now see how many messages we have for our test and train subsets:
print('rows in test set: ' + str(x_test.shape))print('rows in train set: ' + str(x_train.shape))
So we have 1672 messages for testing, and 3900 messages for training!
Our Naïve Bayes model requires data to be in either Tf-IDF vectors or word vector count. The latter is achieved using Count Vectorizer, but we’ll obtain the former through using Tf-IDF Vectorizer.
TF-IDF Vectorizer creates Tf-IDF values for every word in our text messages. Tf-IDF values are computed in a manner that gives a higher value to words appearing less frequently so that words appearing many times due to English syntax don’t overshadow the less frequent yet more meaningful and interesting terms.
lst = x_train.tolist()vectorizer = TfidfVectorizer(input= lst ,  # input is the actual textlowercase=True,      # convert to lowercase before tokenizingstop_words='english' # remove stop words)features_train_transformed = vectorizer.fit_transform(list) #gives tf idf vector for x_trainfeatures_test_transformed  = vectorizer.transform(x_test) #gives tf idf vector for x_test
We fit our Naïve Bayes model, aka MultinomialNB, to our Tf-IDF vector version of x_train, and the true output labels stored in y_train.
from sklearn.naive_bayes import MultinomialNB# train the modelclassifier = MultinomialNB()classifier.fit(features_train_transformed, y_train)
It’s time to pass in our Tf-IDF matrix corresponding to x_test, along with the true output labels (y_test), to find out how well our model did!
First, let’s see the model accuracy:
print("classifier accuracy {:.2f}%".format(classifier.score(features_test_transformed, y_test) * 100))
Our accuracy is great! However, it’s not a great indicator if our model becomes biased. Hence we perform the next step.
Let’s now look at our confusion matrix and f-measure scores to confirm if our model is doing OK or not:
labels = classifier.predict(features_test_transformed)from sklearn.metrics import f1_scorefrom sklearn.metrics import confusion_matrixfrom sklearn.metrics import accuracy_scorefrom sklearn.metrics import classification_reportactual = y_test.tolist()predicted = labelsresults = confusion_matrix(actual, predicted)print('Confusion Matrix :')print(results)print ('Accuracy Score :',accuracy_score(actual, predicted))print ('Report : ')print (classification_report(actual, predicted) )score_2 = f1_score(actual, predicted, average = 'binary')print('F-Measure: %.3f' % score_2)
We have an f-measure score of 0.853, and our confusion matrix shows that our model is making only 61 incorrect classifications. Looks pretty good to me 😊
You can create a heatmap using the seaborn library to visualize your confusion matrix. The code below does just that.
And that’s it to make your very own spam classifier! To summarize, we imported the dataset and visualized it. Then we split it into train/test and converted it into Tf-IDF vectors. Finally, we trained our Naive Bayes model, and saw the results! You could take this a step further and deploy it as a web app if you like.
[1] D. T, Confusion Matrix Visualization (2019), https://medium.com/@dtuk81/confusion-matrix-visualization-fc31e3f30fea
C. Vince, Naive Bayes Spam Classifier (2018), https://www.codeproject.com/Articles/1231994/Naive-Bayes-Spam-ClassifierH. Attri, Feature Extraction using TF-IDF algorithm (2019), https://medium.com/@hritikattri10/feature-extraction-using-tf-idf-algorithm-44eedb37305eA. Bronshtein, Train/Test Split and Cross Validation in Python (2017), https://towardsdatascience.com/train-test-split-and-cross-validation-in-python-80b61beca4b6Dataset: https://www.kaggle.com/uciml/sms-spam-collection-datasetFull code: https://github.com/samimakhan/Spam-Classification-Project/tree/master/Naive-Bayes
C. Vince, Naive Bayes Spam Classifier (2018), https://www.codeproject.com/Articles/1231994/Naive-Bayes-Spam-Classifier
H. Attri, Feature Extraction using TF-IDF algorithm (2019), https://medium.com/@hritikattri10/feature-extraction-using-tf-idf-algorithm-44eedb37305e
A. Bronshtein, Train/Test Split and Cross Validation in Python (2017), https://towardsdatascience.com/train-test-split-and-cross-validation-in-python-80b61beca4b6
Dataset: https://www.kaggle.com/uciml/sms-spam-collection-dataset
Full code: https://github.com/samimakhan/Spam-Classification-Project/tree/master/Naive-Bayes