Supercharge your Python String with TextBlob
By: Khuyen Tran
Tags: Python, Data Science, Data Analysis, Naturallanguageprocessing, NLP

Supercharge your Python String with TextBlob

Get more insights out of your text within one line of code!

Photo by Thomas Kelley on Unsplash

Motivation

Data does not only contain numbers but also text. Knowing how to process text fast will help you to analyze your data faster and gain more insights from your data.

Text processing does not need to be difficult. Wouldn’t it be great if all we need to do to find the sentiment of the text, tokenize text, find word and noun phrase frequencies, or correct spelling is one single line of code? That is when TextBlob comes in handy.

What is TextBlob?

TextBlob aims to provide access to common text-processing operations through a familiar interface. You can treat TextBlob objects as if they were Python strings that learned how to do Natural Language Processing.

NLTK provides some methods to complete those tasks but you might need to call several classes in order to complete different tasks. But with TextBlob, all you need is to use TextBlob(text) in order to access different methods of TextBlob!

Install TextBlob with

pip install -U textblob

python -m textblob.download_corpora

Now all we need to do to supercharge our string is to wrap the text with TextBlob object.

Let’s find out what we can do with our supercharged string.

Word Tokenization

We will borrow some sentences in my article on how to learn data science when life does not give you a break to learn how to use TextBlob.

Word tokenization splits a piece of text into individual words based on certain delimiters.

Instead of splitting our string based on different delimiters such as a “,” or a “.” or space, all we need to tokenize our sentences are blob.words !

WordList(['When', 'I', 'was', 'about', 'to', 'give', 'up', 'I', 'told', 'myself', 'to', 'keep', 'going', 'It', 'is', 'not', 'about', 'working', 'harder', 'it', 'is', 'about', 'working', 'smarter'])

WordList can be used as a Python list. To access the first word, use

>>> blob.words[0]

'When'

Noun Phrase Extraction

A noun phrase is a group of two or more words that center on a noun (e.g., ‘dog’, ‘girl’, ‘man’) and include modifiers (e.g., ‘the’, ‘a’, ‘None of’). For example, ‘girl’ is not a noun phrase but ‘a beautiful girl’ is a noun phrase.

Sometimes, it is important for us to extract all noun phrases in the sentence instead of extracting individual nouns. TextBlob allows us to do that easily

WordList(['learning strategies'])

As we can see, just ‘learning strategies’ are extracted from the sentence because it is the only noun phrase in the sentence.

Sentiment Analysis

We can also extract the sentiment of a sentence using blob.sentiment

Sentiment(polarity=0.15833333333333333, subjectivity=0.48333333333333334)

Polarity is a float that lies within the range of (-1,1). If the polarity is below 0, the sentence is more negative than positive. If the polarity is above 0, the sentence is more positive than negative. Since our polarity is 0.15, it is more positive than negative.

Subjectivity refers to personal opinion. Subjectivity is a float that lies in the range (0,1). If the value of subjectivity is above 0.5, the sentence is more subjective than objective and vice versa. Since the subjectivity of the sentence is 0.48, it is more objective than subjective.

Lemmatization

Lemmatization is the dictionary form of a set of words. For example, ‘eats’, ‘ate’, ‘eating’ are all forms of the word ‘eat’. Lemmatization is a popular method to process text before applying other NLP models.

In order to lemmatize word, wrap the word inside the Word object instead of TextBlob

know

that

i

can

complete

these

things

in

a

short

amount

of

time

make

me

excite

and

be

in

the

flow

Awesome! Some words are converted into their dictionary form. ‘knowing’ is converted ‘know’, ‘excited’ is converted to ‘excite’, and ‘makes’ is converted to ‘make’. Now the NLP model will not regard ‘knowing’ and ‘know’ as 2 different words!

Spelling Correction

Sometimes, we might find a misspell or two in an article. The misspell is even more common if we are analyzing social media’s text such as Twitter. The misspelling can make it difficult for our NLP model to know that the word learnin and the word learning are basically the same words!

TextBlob also allows us to correct the spelling with one line of code

I intentionally misspell a couple of words in the sentence such as neded, swicth, and learnin. Let’s see how TextBlob correct these words

TextBlob("I just need to switch my learning strategics.")

Pretty cool! All the misspelling is corrected in a reasonable way.

We can also correct individual words with Word().spellcheck()

[('learning', 0.25384615384615383),

('warning', 0.24615384615384617),

('margin', 0.2153846153846154),

('latin', 0.13076923076923078),

('martin', 0.05384615384615385),

('lain', 0.046153846153846156),

('earning', 0.038461538461538464),

('marin', 0.007692307692307693),

('darwin', 0.007692307692307693)]

The misspelling word lanin can be the misspell of many things so TextBlob gives us different results with learning is the word with the highest percentage to be the right one.

Word Frequencies

Sometimes, it is important to know the frequency of a certain word in a sentence. TextBlob allows us to find the frequency of a word in a text with ease.

We will use all of the text in my article on how to learn data science when life does not give you a break. We will use newspaper to scrape the article.

pip install newspaper

Since this article is about my data science journey, I anticipate that there will be a lot of the words “I” in the sentence. Let’s check that with blob.word_counts

81

81 is a high number but what how high it is compared to other words in my article? Let’s visualize it with bar plot

As we could guess, words such as ‘i’, ‘to’, ‘the’, ‘my’ are among the most common words in the article!

Conclusion

Congratulations! You have just learned how to process your Python string with TextBlob. And the only step needed to use all processing methods listed above is to wrap the string inside the TextBlob object or the Word object. How easy it is!

You can explore more methods to process and analyze your text in TextBlob’s documentation.

The code for this article could be found here

I like to write about basic data science concepts and play with different algorithms and data science tools. You could connect with me on LinkedIn and Twitter.

Star this repo if you want to check out the codes for all of the articles I have written. Follow me on Medium to stay informed with my latest data science articles like these: