Natural Language Processing: Sentiment Analysis and the TextBlob Library
By: Ajay S
Tags: Naturallanguageprocessing, Textblob, Sentiment Analysis, Social Media, NLP

The Merriam Webster Dictionary defines the word “Artificial” as “humanly contrived, man-made; often on a natural model” and “Intelligence” as “the ability to learn or understand or to deal with new or trying situations.” So, quite literally “Artificial Intelligence” translates to a man-made model capable of learning. Artificial Intelligence focuses on giving machines a human-like viewpoint in solving problems. AI is a quintessential component of modern technology. Its applications extend in a vast spectrum: from social media, business analytics and medical examinations to government security systems.

AI focuses on building computer programs that can access data so that the model can learn by itself without being explicitly programmed. It tries to provide systems the capability to automatically learn from experience and improve without human intervention. In Artificial Intelligence, we try to simulate human like intelligence into machines so that they try to mimic human like actions. This problem solving is applied across various sectors such as banking, marketing, finance, agriculture, healthcare, gaming, social media, space exploration, autonomous vehicles and so on. The entire global economy is dependent on rise of artificial intelligence at this moment and will only continue to depend more in the future.

Machine Learning connects the gap between man and machine.

Natural Language Processing (NLP) is the meeting point of computer science, Machine Learning algorithms and language.

The purpose of NLP is to train machines to understand and process natural language, so that it can perform human like tasks like translation, interpretation and so on, without actual human intervention. Currently, NLP is one of the leading technologies in the world, with the increasing demand of sentiment interpreters and chatbots. The areas of application of NLP range from the most basic tasks to the most involved projects, such as:

· Speech-to-text, translators, speech recognition, virtual voice assistants like Siri, Google, Alexa and so on.

· Personalized/corporate chat bots/virtual agents for assistance and customer care

· Spell check, grammar check, advanced searching and other language functions

· Data retrieval and information extraction, web scrapping and interpretation

· Recommendation systems, sentiment interpretation for marketing/outreach, fraud detection

The up-and-rising field called Sentiment Analysis is an application of NLP-AI models that attempt to understand emotions through text and provide insights in a responsive way. This sub-field of Natural Language Processing is the interpretation and classification of emotions (positive, negative and neutral) within text data using text analysis techniques. Sentiment analysis allows businesses to identify customer sentiment toward products, brands or services in online conversations and feedback.

These models try to detect and categorize the “polarity” within a text (a positive or negative thought), regardless of whether it’s a whole document, a paragraph, a sentence, a clause, a comment or a Tweet. Understanding the comments and emotions is now an integral part of marketing, even though it is not widely realized by many. Social media analysis is of great importance for marketing strategy and related research. Facebook, Amazon, Google and other technology giants use Machine Learning and Artificial Intelligence models to personalize data for their users. Amazon uses these models for recommendation systems for their customers.

The sentiment of the customers give valuable insights to the company for deciding strategies.

In essence, Sentiment Analysis is basically mining or discovering meaningful and relevant pattern in text data.

For example, if companies use a sentiment interpreter to understand the viewpoints and feedback of customers, they can further enhance their insights and tune their work accordingly. Data scientists and business analysts utilize sentiment analysis to interpret information for researching the latest trends in the market, track the market moves, understand customer experience and so on. This process is very beneficial to understand what exactly the customer of a brand expects from the company.

These methods act as handy tools for the data analysts in a company to map out blueprints.

The process of sentiment analysis is quite involved: first, the text data is “tokenized”: a process that breaks up the sentences into individual words. After tokenization, unwanted words in the data, i.e., words which do not contribute much in understanding the sentiment value of the data, are removed. These words are called “stopwords” and a comprehensive list of stopwords is often used by economic analysts in ease of operation. After this, a process called “lemmatization” is done, where the words are transformed into their original form by removing the grammatical usage. For example, the word “stop” is replaced in place of words like “stopped”, “stopping” and so on.

A very common library used for these operations is “TextBlob”. TextBlob is a library very common among data analysts since it provides a simple API for diving into common natural language processing (NLP) tasks such as part-of-speech tagging, noun phrase extraction, sentiment analysis, classification, translation, and more. It solves several NLP tasks such as summarization, translation, spell and grammar check and so on. The sentiment function of TextBlob returns two properties namely polarity and subjectivity. Polarity is a float that lies in the range [-1, 1] where 1 indicates positivity and -1 indicates negativity. Subjective sentences refer to whether the statement is a personal opinion or emotion whereas an objective statement generally denotes factual information. TextBlob provides in-built classifiers using which a custom classifier can be created.

For example, let’s consider the popular “USComments” YouTube video dataset that contains the comments from a top trending video as shown below:

The USComments.csv dataset contains almost 700k comments from top trending YouTube videos in the US and UK.

The dataset contains almost 700,000 comments from users across the world. After deploying the TextBlob library on this data, the results were seen as follows: