3 Super Simple Projects to Learn Natural Language Processing using Python | by Eric Kleppen | Towards Data Science

Working in Data Science and having a background in Technical Writing, I was drawn to the field of Natural Language Processing (NLP). Machines understanding language fascinates me, and I often ponder which algorithms Aristotle would have used to build a rhetorical analysis machine if he had the chance. If you’re new to Data Science, getting into NLP can seem complicated, especially since there have been so many recent advancements in the field. It is hard to know where to start.
These three super simple projects will give you an introduction to concepts and techniques used in Natural Language Processing.
Word CloudSentiment AnalysisSpam Detection
The data used for these projects is the spam email data set, and it can be found with all of the code in my GitHub:
github.com
While a computer can actually be quite good at finding patterns and summarizing documents, it must transform words into numbers before making sense of them. This transformation is needed because machines “learn” thanks to mathematics, and math doesn’t work very well on words. Before transforming the words into numbers, they are often cleaned of things like special characters and punctuation, and modified into forms that make them more uniform and interpretable.
Cleaning the words is often called preprocessing, and that is the focus of project 1: Word Cloud.
Start by importing the dependencies and the data. The data is stored as a comma separated values (csv) file, so I will use pandas’ read_csv() function to open it into a DataFrame.
import pandas as pdimport sqlite3import regex as reimport matplotlib.pyplot as pltfrom wordcloud import WordCloud#create dataframe from csvdf = pd.read_csv('emails.csv')df.head()
Before anything, it is best to do a quick analysis of the data to eliminate duplicate rows and establish some baseline counts. I use pandas drop_duplicates to drop the duplicate rows.
print("spam count: " +str(len(df.loc[df.spam==1])))print("not spam count: " +str(len(df.loc[df.spam==0])))print(df.shape)df['spam'] = df['spam'].astype(int)df = df.drop_duplicates()df = df.reset_index(inplace = False)[['text','spam']]print(df.shape)
Word clouds are a useful way to visualize text data because they make understanding word frequencies easier. Words that appear more frequently within the email text appear larger in the cloud. Word Clouds make it easy to identify “key words.”
Notice in the word cloud image, all the text is lower case. There are no punctuation marks or special characters. That’s because the text has been cleaned to make it easier to analyze. Using regular expressions, it is easy to clean the text using a loop:
clean_desc = []for w in range(len(df.text)):    desc = df['text'][w].lower()        #remove punctuation    desc = re.sub('[^a-zA-Z]', ' ', desc)        #remove tags    desc=re.sub("&lt;/?.*?&gt;"," &lt;&gt; ",desc)        #remove digits and special chars    desc=re.sub("(\\d|\\W)+"," ",desc)        clean_desc.append(desc)#assign the cleaned descriptions to the data framedf['text'] = clean_desc        df.head(3)
Notice I create an empty list clean_desc, then use a for loop to go through the text line by line, setting it to lower case, removing punctuation and special chars, and appending it to the list. Then I replace the text column with the data in the clean_desc list.
Stop words are the most common words like “the” and “of.” Removing them from the email text allows the more relevant frequent words to stand out. Removing stop words is a common technique! Some Python libraries like NLTK come pre-loaded with a list of stop words, but it is easy to create one from scratch.
stop_words = ['is','you','your','and', 'the', 'to', 'from', 'or', 'I', 'for', 'do', 'get', 'not', 'here', 'in', 'im', 'have', 'on', 're', 'new', 'subject']
Notice I include a few email related words like “re” and “subject.” It is up to the analyst to determine what words should be included or excluded. Sometimes it is beneficial to include all words!
Conveniently there is a Python library for creating word clouds. It can be installed using pip.
pip install wordcloud
When constructing the word cloud, it is possible to set several parameters like height and width, stop words, and max words. It is even possible to shape it instead of displaying the default rectangle.
wordcloud = WordCloud(width = 800, height = 800, background_color = 'black', stopwords = stop_words, max_words = 1000                      , min_font_size = 20).generate(str(df1['text']))#plot the word cloudfig = plt.figure(figsize = (8,8), facecolor = None)plt.imshow(wordcloud)plt.axis('off')plt.show()
The word cloud can be saved and displayed using matplotlib and .show(). This is the result of all records, regardless of it being spam.
Push the exercise further by splitting the data frame and making two word clouds to help analyze the difference between key words used in spam email and not spam email.
This is a binary classification problem since an email can either be spam (1) or not spam (0). I want to build a machine learneing model that can identify whether or not an email is spam. I am going to use the Python library Scikit-Learn to explore tokenization, vectorization, and statistical classification algorithms.
Import the Scikit-Learn functionality we need to transform and model the data. I will use CountVectorizer, train_test_split, ensemble models, and a couple metrics.
from sklearn.feature_extraction.text import CountVectorizerfrom sklearn.model_selection import train_test_splitfrom sklearn import ensemble from sklearn.metrics import classification_report, accuracy_score
In project 1, the text was cleaned. When you look at a word cloud, notice it is primarily single words. The larger the word, the higher its frequency. To prevent the word cloud from outputting sentences, the text goes through a processes called tokenization. It is the process of breaking down a sentence into the individual words. The individual words are called tokens.
Using SciKit-Learn’s CountVectorizer(), it is easy to transform the body of text into a sparse matrix of numbers that the computer can pass to machine learning algorithms. To simplify the concept of count vectorization, imagine you have two sentences:
The dog is whiteThe cat is black
Converting the sentences to a vector space model would transform them in such a way that looks at the words in all sentences, and then represents the words in the sentence with a number.
The dog cat is white blackThe dog is white = [1,1,0,1,1,0]The cat is black = [1,0,1,1,0,1]
We can show this using code as well. I’ll add a third sentence to show that it counts the tokens.
#list of sentencestext = ["the dog is white", "the cat is black", "the cat and the dog are friends"]#instantiate the classcv = CountVectorizer()#tokenize and build vocabcv.fit(text)print(cv.vocabulary_)#transform the textvector = cv.transform(text)print(vector.toarray())
Notice in the last vector, you can see a 2 since the word “the” appears twice. The CountVectorizer is counting the tokens and allowing me to construct the sparse matrix containing the transformed words to numbers.
Because the model doesn’t take word placement into account, and instead mixes the words up as if they were tiles in a scrabble game, this is called the bag of words method. I am going to create the sparse matrix, then split the data using sk-learn train_test_split().
text_vec = CountVectorizer().fit_transform(df['text'])X_train, X_test, y_train, y_test = train_test_split(text_vec, df['spam'], test_size = 0.45, random_state = 42, shuffle = True)
Notice I set the sparse matrix text_vec to X and the df[‘spam’] column to Y. I shuffle and take a test size of 45%.
I highly recommend experimenting with several classifiers and determine which one works best for this scenario. In this example, I am using the GradientBoostingClassifier() model from the Scikit-Learn Ensemble collection.
classifier = ensemble.GradientBoostingClassifier(    n_estimators = 100, #how many decision trees to build    learning_rate = 0.5, #learning rate    max_depth = 6)
Each algorithm will have its own set of parameters you can tweak. That is called hyper-parameter tuning. Go through the documentation to learn more about each of the parameters used in the models.
Finally, we fit the data, call predict and generate the classification report. Using classification_report(), it is easy to build a text report showing the main classification metrics.
classifier.fit(X_train, y_train)predictions = classifier.predict(X_test)print(classification_report(y_test, predictions))
Notice our model achieved 97% accuracy. Push the exercise further by tweaking the hyper-parameters, exploring different classifiers, and trying different vectorizers!
Sentiment Analysis is also a classification problem of sorts. The text is essentially going to reflect a positive, neutral, or negative sentiment. That is referred to as the polarity of the text. It is also possible to gauge and account for the subjectivity of the text! There are a ton of great resources that cover the theory behind sentiment analysis.
Instead of building another model, this project uses a simple, out of box tool to analyze sentiment called TextBlob. I’ll use TextBlob to add sentiment columns to the DataFrame so it can be analyzed.
Built on top of NLTK and pattern, the TextBlob library for Python 2 and 3 tries to simplify several text processing tasks. It provides tools for classification, part-of-speech tagging, noun phrase extraction, sentiment analysis and more. Install it using pip and check out the installation guide.
pip install -U textblobpython -m textblob.download_corpora
Using the sentiment property, TextBlob returns a named tuple of the form Sentiment(polarity, subjectivity). Polarity is a float in the range [-1.0, 1.0] where -1 is the most negative and 1 is the most positive. Subjectivity is a float in the range [0.0, 1.0] where 0.0 is very objective and 1.0 is very subjective.
blob = TextBlob("This is a good example of a TextBlob")print(blob)blob.sentiment#Sentiment(polarity=0.7, subjectivity=0.6000000000000001)
Using list comprehensions, it is easy to load the text column as a TextBlob, and then create two new columns to store the Polarity and Subjectivity.
#load the descriptions into textblobemail_blob = [TextBlob(text) for text in df['text']]#add the sentiment metrics to the dataframedf['tb_Pol'] = [b.sentiment.polarity for b in email_blob]df['tb_Subj'] = [b.sentiment.subjectivity for b in email_blob]#show dataframedf.head(3)
TextBlob makes it super simple to generate a baseline sentiment score for polarity and subjectivity. To push this exerciser further, see if you can add these new features to the spam detection model to increase the accuracy!
Even though Natural Language Processing can seem like an intimidating topic, the foundational pieces are not that hard to grasp. There are plenty of libraries that make it easy to begin exploring data science and NLP. Completing these three projects:
Word CloudSpam DetectionSentiment Analysis
You explore concrete examples of applying preprocessing, tokenization, vectorization, and feature engineering on text data.
If you’re interested in learning more about data science or programming, check out my other articles!
towardsdatascience.com
If you enjoyed this, follow me on Medium for more
Get FULL ACCESS and help support my content by subscribing
Let’s connect on LinkedIn
Analyze Data using Python? Check out my website
— Eric Kleppen