NAIVE BAYES SPAM FILTER - SRT521 Project Phase 1 and 2
Channel: Alberto Munar
Date: 2024-01-01

Naïve Bayes Spam Filter: Step by Step
 Write a python program to do the following task:
 Along with the program, you need to properly define each of the steps and provide
complete detail in the report. Report should also include instructions on executing
your program. Your code should be well-documented internally as well.
 You must develop the code by yourself. However, you can make use of standard python
libraries.
 You may borrow ideas from other packages available on the web. You must cite and give
full details (URL, authors' names) to any ideas that you incorporate.
1. Data Preparation
 Load the data into python and do the necessary conversions.
 Look for any missing values and get rid of those records.
 Prepare some graphs to perform text analytics on your data and to show other
useful values.
2. Feature Transformation
 Right now, your dataset contains 2 features (subject and message).For spam
filters, you need to have individual words along with their frequencies. Perform
the necessary operations needed such as extracting individual tokens, remove
punctuation and all the stop words.
 You can also make use of any NLP text classification model such as BERT (this is
just one model, you can explore any one from fast text, word2Vec to work on .
Check the NLTK platform for more on natural language processing.
 Check working with text data in Scikit-Learn.
 Split the data into training and test datasets.
3. Train the Model
 Use the given data to train your model.
 Your program should process the subject and email message and decide the values
for label to be “0”(ham) or “1”(spam).
4. Test the Model
 Test your model on the test data and compare the results by printing the predicted
and actual values and finally by printing the model accuracy.
 Plot the percentage of spam and hams identified correctly versus incorrectly.
4 | P a g e
SRT521 PROJECT
5. Test the Model on New Data
 Let the program read a new email from the command prompt and try to label it.
 The output of this module should be a message saying that the given email is spam
or not.
 Data should then be added to the training model and perform step 4 again.
 Compare the result of this phase with the last one to see if your model is improving
with new data or not.
6. Bonus
 Your program should implement some form of adaptation so that more recent
training examples are given higher weight than older ones.
 Your program should provide some way for the user to adjust the tradeoff
between false positive and false negative errors. This could be done either
through a command line argument or through a configuration file.
Your work will be graded on the following criteria:
• Functionality: Does the program execute successfully and filter emails? [120 marks]
• Code Quality and Documentation: Is the program well-documented (internally and
externally)? [40 marks]
• Accuracy :How accurate is the output ? [40 marks]
Submission Requirement
The project is to be submitted in two phases:
Phase 1 Submission
 Complete task 1 and 2
 Files Required for Submission:
o Python code for data preparation and feature transformation.
o Data set in csv format.
o Report is pdf format outlining the research and explaining how you have
accomplished tasks 1 and 2 and the plan for the remaining tasks.
o The report should also provide details on work distribution, the role of
each team member and the tasks completed by them.
o Properly cite all resources.
 Phase 1: Demonstration will be in class on Monday, 13 th November 2023 and all
required files should be submitted by 15 th November 2023 @ 11:59 pm
5 | P a g e
SRT521 PROJECT
Phase 2 Submission
 Complete task 3-5 and bonus if attempted.
 Files Required for Submission:
o The complete python code includes tasks 1 and 2.
o Data set in csv format.
o Report is pdf format outlining the research and explaining how you have
accomplished all the tasks for this phase.
o The report should also provide details on work distribution, the role of each
team member and the tasks completed by them.
o Properly cite all resour