Let’s Learn Sentiment Analysis
By: Praffulla Dubey
Tags: Analysis, NLP, Sentiment Analysis, Sentiment, Naturallanguageprocessing

Photo by Lukas Blazek on Unsplash

Introduction

In recent times, it has become a trend to find the emotion behind a person’s words. Various companies are taking the customer feedback very seriously. They are using various machine learning algorithms to find the customer’s tone in their feedback. This method of finding the tone of one’s speech or feedback can be termed as emotion AI, opinion mining, or sentiment analysis.

Sentiment analysis is a powerful tool that can be used to understand the tone of a piece of writing. It is a tedious task for humans to do, but with the help of machine learning this has become an easy task.

With the help of sentiment analysis, we can classify the human speech into various categories like positive, negative, neutral, very positive and very negative. With the help of Machine Learning, Natural Language Processing (NLP), Text Mining and Lexicon based approach we can extract these features associated with sentiment analysis.

Lexicon based approach uses the concepts of natural language processing (NLP) and the dictionary data structure to extract the tone of the words in the speech. Whereas, we need a sentiment labelled collection of documents if we use machine learning. We can use supervised machine learning algorithms to predict the sentiment of the speech.

Python has a vast set of libraries and we can use them for finding the tone of a passage or text. We can use nltk (natural language toolkit), SpaCy, TextBlob, Gensim, etc. In this research I propose to use SpaCy.

Photo by Mark Rasmuson on Unsplash

Proposed Work

We follow the Data Science Pipeline for our model. The very first step is to load the data that we need to work on with and this step is known as Data Gathering. After data gathering, we need to clean the data as the data is collected from various sources and the data may be dirty. The data can also contain various null values and it can affect our prediction model. Hence, we need to clean this data and this step is known as Data Cleaning. After data cleaning we analyse the data with the help of various graphs. The step of analysing the data is known as Data Analysis and the step of creating various graphs is known as Data Visualisation. After this we split our dataset into training and testing data and then the training data is used to train our model and the testing data is used to test our model.

As we are dealing with text data for sentiment analysis, therefore after loading the data we need to process the data with the help of the natural language processing pipeline. The necessary steps of natural language processing pipeline include the following:

Breaking down of text into words, sentences or other units known as Tokenizing. The text contains various unrequired words like I, me, myself, very, etc. These words are termed as stopwords and they are not required for our model. Hence we remove these stopwords and we can term this process as Removal of Stopwords. We condense all forms of words into a single form and this step is known as Normalizing. For the classification purpose we need to convert the text into numerical form and this step is known as Vectorization.

After performing the above mentioned step, the data we get is clean data.

Using SpaCy for Data Preprocessing

Photo by NASA on Unsplash

SpaCy is an open-source software library developed for advance natural language processing and it is written in Python and Cython programming languages. It is widely used and it also supports deep learning workflow. Some of the main features of SpaCy are named entity recognition, statistical models for eleven languages, part of speech (POS) tagging, pre-trained words vectors, text classification and deep learning integration.

The very first step for finding the tone of our text we need to install the required libraries here, SpaCy. It can be downloaded with the pip command as shown:

“pip install spacy”

Secondly, we tokenize the text with the help of spaCy. We can do Word Tokenization (breaking the text into words) or Sentence Tokenization (breaking the text into sentences). There are 11 languages present in spaCy, therefore we need to load the languages we need to work on with.

We can say that stop words are an important part for human speech but for machines, stop words are just waste as they do not play any important part in the NLP tasks. There is a default list of words in the spaCy library that are considered as stop words. There is an option for modifying this list and we can add new words to the list of stop words. To remove the stop words from the filtered text we can use “.is_stop” keyword.

Next step is to normalize the words. We should bring all the words to their root form like, “walking” and “walked” should be represented by a single word i.e. “walk.” This can be done by Stemming (it cuts the words to create the smallest units of words) and Lemmatization (it uses a data structure that relates all forms of words back to its simplest form which is known as lemma). As lemmatization is more powerful than stemming therefore it is the only technique provided by spaCy for normalization purposes.

Now to make our machine understand our text we need to convert it to machine understandable form and this is known as vectorization. Vectorization converts the tokens into vectors or an array that is used to represent the various features that a token contains. With the help of vectors, we can classify the text or apply various other NLP operations.

We can use spaCy to text classification. It provides “textcat” (TextCategorizer), one of the built-in pipeline components of spaCy that enables the user to specify or assign labels or categories to the text data and then use it as a training data. We just need to add valid labels for the textcat component, then load, shuffle and split the data and then use this data to train the model.

Conclusion

NLP is a large field and we have many different approaches for doing different things. Therefore, the deeper we research, the deeper we learn. Also, we can use various other algorithms and libraries like BERT or VADER to predict the tone or sentiment of the text.

References