NLP Mastery Part -1
By: Himanshu Tripathi
Tags: Feature Engineering, Coding, Data Science, Machine Learning, Naturallanguageprocessing

Photo by Joshua Aragon on Unsplash

In this series of articles, we’re not just going to learn about how to use NLP to solve real-world problems, but we’re also going to learn about the Maths behind these concepts. So that we can understand how these algorithms actually work.

Natural language processing (NLP) is a subfield of linguistics, computer science, and artificial intelligence concerned with the interactions between computers and human language, in particular how to program computers to process and analyze large amounts of natural language data. The result is a computer capable of “understanding” the contents of documents, including the contextual nuances of the language within them. The technology can then accurately extract information and insights contained in the documents as well as categorize and organize the documents themselves. (Source:- wiki)

Sounds complex right? Let’s try to break it down into a simple way

We all have heard about or used any one of these Google’s Assistant, Apple Siri, or Amazon Alexa but they didn’t speak your language, and they couldn’t understand you? So how they can understand us and how they can answer all of our questions?

Or you go to a new place and you don’t know their languages but by using Translator you can simply translate it? So how this all comes?

Here comes the magic of NLP (Natual Language Processing)

Vocabulary and Feature Extraction

In this article, we’re going to learn about How to represent a text as a vector.

For doing this we need to build a vocabulary, but wait what is vocabulary?

So Vocabulary is nothings just a list of unique words from a given list of text corpus, to get the list first we have to go through all the words from all the given text corpus and save every new word that appears in the search. (simple isn’t)

Let’s Take an example

Given text corpus is :

Making Vocabulary out of it :

As you can see there is no duplicate values only unique value

Now let’s take new text and extract features using our vocabulary. To do so we have to check if every word from our vocabulary appears in the text corpse. If it does then assign a value 1 else 0.

So as we can see we have 1 for those words that are present in our Vocabulary and 0 for those which are not in the vocabulary. So we have six ones and many zeros. For now, our vocabulary is very small we don’t have many zeros but in the real world, we have a lot of words in our vocabulary so in that case, we have lots of zeros, Now this type of representation with a small relative number of non-zeros values is called a sparse representation.

Problem with Sparse Representations

Our features_extraction has a size that is equal to the size of our vocabulary but here is most of the values are zeros which doesn’t make any sense. So our model let’s say (Logistic Regression) has to learn n+1 parameters, where n is the size of the vocabulary. So you can imagine that for the large vocabulary which we have in the real world this would be problematic.

So for 10000 words or vocabulary, we have 10000 + 1 parameters to learn which can lead to an excessive amount of time to train and much more time to making the prediction.

Positive and Negative Frequencies

As as we’ve seen above we have a problem of sparse representations which leads to more training time and more time for making the prediction, So how are we going to solve this issue?

In reality, we want to keep track of the number of times it shows up as a positive class and the number of times that word showed up in the negative class. Using both those counts, we can then extract features and use those features in any classification model and train our model.

Let’s say we have a corpus

And Vocabulary

For this particular example, we can see we have two classes. One associated with positive sentiment and the other with negative sentiment.

Positive Sentence

Negative Sentence

Positive Frequency Count

Negative Frequency count

Word Frequency in classes

So this is the entire table with the positive and negative frequencies for the corpus. This table is a dictionary mapping from a word-class there to its frequency.

Now we’re going to use a frequency dictionary to represent a text. We are going to encode a text represented as a vector of dimension 3. So what do I mean about dimension 3?

So take an arbitrary tweet m. The first feature would be a bias unit equal to 1. The second feature is the sum of the positive frequencies for every unique word on text m. The third is the sum of the negative frequencies for every unique word on the text.

Sum of positive frequencies:- 10

Sum of Negative frequencies:- 8

Hence we end up getting the following features vector [1,10,8]. 1 corresponds to the bias, 10 the positive feature, and 8 the negative feature.

So for now, we know how to represent the given text into vector from. So In the next article, we’re going to learn about how to process the text for the model building.

That’s it for now 👏👏. See you in the next Article.

Reference :- https://www.coursera.org/specializations/natural-language-processing

Check out more interesting Machine Learning, Deep Learning, Data Science Projects on my YouTube 👉 :- YouTube ( 👍)

Check out my previous articles:

If you found this article interesting, helpful and if you learn something from this article, please clap(👏👏) and leave feedback.

Thanks for reading!