Building Things — Context Based Sentiment Analysis in Product Review Videos
By: Kumar Vidit
Tags: Sentiment Analysis, Product Marketing, Product Design, Machine Learning

The word “Social” has taken a whole new meaning in today’s digital era. Simply going out to enjoy is no longer the only “social” criteria. Social now is — giving a peek in your personal and professional life to your connections. Facebook, Twitter, Instagram, and other leading platforms have connected people in ways that were unimaginable 20 years ago. More than that, these platforms have become an excellent resource for businesses to reach new customers, understand the perception of their brand, seek feedback, and improve customer experience. Today, analytics has become a driving force for businesses, enabling them to use the power of data to grow user base and revenues.

What exactly is Sentiment Analysis?

One such power, Sentiment Analysis — a concept popular amongst Machine Learning enthusiasts — is leveraged by companies to understand customer sentiment or emotion regarding the company’s products. Such an analysis crawls social media platforms to gather data on user sentiments on specific products by the company. It analyzes each user’s comment to classify it as positive, negative, or neutral, and to provide the overall result. The market has witnessed an exponential growth since 2016.

Customer Experience & Product/Market Research categories will continue to see the highest growth rates

Where the traditional approach falls apart!!

“Without context a piece of information is just a dot. It floats in your brain with a lot of other dots and doesn’t mean a damn thing. Knowledge is information-in-context..connecting the dots” — Michael Ventura

I see Sentiment Analysis as a powerful technique that is yet to be fully tap-into. The current textual analysis has many shortcomings. “Yeah, no one does it better than you!”. The software will classify it as a positive sentiment. But what if it was Sarcasm? Here is another one. “…Amazon always does it”. What should I make of this comment? Let’s take a step back in the text and see a sentence before this. “Amazon is great in delivering products on time. Amazon always does it” — makes the latter sentence positive. However, “Amazon just delivered a damaged piece. Amazon always does it”- makes the text negative. Context is very crucial in understanding a sentiment. Text analysis has been missing the context of the conversation!

Another problem is — we have been analyzing only textual data. There are more channels to explore. “iPhone 11 pro complete review” — a user searches on YouTube. From a teenager to an adult, everyone searches for a product review on the internet before buying it. “This is my review video on the new Bose’s Noise Cancellation headphones” — a passionate technology user on Twitter. Defeating word of mouth, word of such videos has become a direct influencer of people’s buying decisions. These videos are an excellent resource for businesses to seek user sentiment and feedback on their products.

Welcome to the new age of context sentiment analysis — My approach

In this article, I suggest a new approach towards Sentiment Analysis — Context-based hierarchical analysis of videos uploaded by product users. I decided to analyze not only text but also visuals and audio by extracting important attributes — facial expressions, speech tone, and voice intensity. To capture the context, we will analyze the smallest unit of speech separated by pauses — ‘’utterance’’. We want each utterance to seek information from the previous and the next utterance. Bi-directional Long short-term memory (LSTM) serves such requirements. A side benefit of LSTM– it resolved the vanishing/exploding gradient issue that I feared the network would face while learning long term dependencies. We want to analyze not only the video frames but also the changes in consecutive frames. 3D- Convolutional Neural Network was built just for the job!

Tackling one problem at a time, I developed the following algorithm:

1. Extract Features for Text, Audio, and Video for each utterance

Text features from transcripts of spoken words using Convolutional Neural Network

Audio features by using an open-source tool like OpenSMile

Visual feature extraction using 3D-CNN

2. For each channel (Text, Audio, and Video)

Send the extracted features through a Bidirectional Long short-term memory (bi-LSTM) Neural Network to obtain context incorporating features

3. Append context features of Text, Audio, and Video channels and feed to an LSTM network

4. Send the output to a dense layer and then SoftMax layer for classification, using categorical cross-entropy on utterance’s SoftMax output for training

5. After the training phase, pass the test set through the network to get context incorporating features

6. Feed those features through SVM for classification

The above approach incorporates the shortcomings of traditional text sentiment analysis and achieves a better accuracy of ~80% on MOSI data, which contains video reviews annotated by sentiment polarity.

Value creation for businesses

A) To make a more informed decision regarding your brand and products

Insightful sentiment analysis eliminates the guesswork involved in evaluating the performance of your products. Based on the insights, you can adjust to the current market needs, and increase your customer satisfaction. With such data in hand, precisely calculating customer-retention becomes easier. You can also use sentiment analysis to evaluate a new product concept before bringing it to life by putting the idea through concept testing and analyzing customer sentiment.

B) To gain a competitive edge in the market

Run the tool to get sentiments on your competitor’s products. Such knowledge will act as an incentive to keep up with the market and boost the performance of your products. It can also help you realize consumer trends early and leverage the same to get the edge in the market.

C) To enhance customer experience

Many consumers share their experiences with the internet community through their online feedback. Their tone and temperament can be identified and labeled as a positive, negative, or neutral sentiment. So, you can know what is correctly implemented in your products, and what needs further improvement.